Dobject building incomplete if more that 5 mio topics

Issue description

The logarchive system is now at 7 mio topics and 20 mio association and occurrences each. After building the dobjects the actual number of dobjects in the directory was only 5 Mio. Most likely the system ran out of disk space since the dobjects are build in a new directory D1 before swapping into the correct location.

Developer comments

At 9.5 mio topics it is currently impossible to build the dobjects. The system kills the respective process and the stressful load is visible for users of neighbor system on the server. They become unresponsive and users report that saving does not work and pages are not displayed all together.

Introduced options %min% and %max% for %build-dobjects% in order to allow building of the dobjects not in one single process, but rather in tranches.

Tranches works, but the building of 5 mio topics takes 30 minutes. Most of the time is spent writing the dobjects to the filesystem (25 min). This could be easily parallelized since the computation is already completely finished.

Affects

topincs build-dobjects(Command)

Reporting date

2026-03-13

Reported by

Robert Cerny

Resolving date

2026-04-15

Resolved by

Robert Cerny

Resolved in version

Topincs 10.1.0 – next release

Part of

Big Data(Enhancement)

Work sessions

Start	2026-03-24T16:13:38
End	2026-03-24T18:24:39
Participant	Robert Cerny