Over the last few days, there was a flood of spam to the CST Wiki. That was causing two problems:
- I had to constantly promote the good pages over the spam pages
- The wiki implementation keeps all versions of all pages in memory - so it was getting bigger
To solve the first problem, I added a simple blacklist. The spam that hits the Wiki seems to come in waves, against a bunch of pages, and all of the same kind. So adding a specific filter to the blacklist solves the problem. The second problem was something I suspected, but didn't verify until this evening. After loading all pages into memory, the system was running right into the (system imposed) limits on memory. So that meant that it was getting into thrashing - it always needed more memory, but the internal limits wouldn't allow it to.
As it happens, that's easy to fix in Cincom Smalltalk. First, you can raise the upper bounds on memory, at runtime, like this:
ObjectMemory currentMemoryPolicy memoryUpperBound: someDesirableNumberHere.
That solves part of the problem. The other part is ensuring that the system has big enough memory zones right at startup, so that it doesn't have to thrash to get there. That's accomplished by sending a message like this:
ObjectMemory sizesAtStartup: #("array of seven numbers goes here")
The documentation for that method is probably called for here:
"Sets the suggested initial sizes of various memory spaces for future snapshots. The spaces whose sizes can be set are as follows:
- Eden
- SurvivorSpace
- LargeSpace
- StackSpace
- CompiledCodeCache
- OldSpaceHeadroom
- FixedSpaceHeadroom
These sizes are suggestions only, the virtual machine may have to deviate
from these settings in order to successfully load an image. In general, the
sizes are specified as a multiple of the platform-dependent default
value for each space. In other words, if the first element of arrayOfSizes is
1.5, then Eden's suggested size will be 1.5 times the default size of Eden
on the platform that the system is being started up on. A value of 1.0,
on the other hand, will result in the default value being used. A value
of nil in a particular slot of the arrayOfSizes, will have no effect on the
current setting for that particular space. Fails if the arrayOfSizes is not as
large as the number of spaces whose initial sizes can be specified or if the
values in the arrayOfSizes are not either nils or Floats that are greater than
or equal to zero and less than or equal to 1000. Returns the receiver."
The important one in the case of the Wiki was number 6 - old space. Any runtime objects that survive initial GC end up in old space, so making it bigger solved the thrashing problem. Of course, you could argue that the Wiki shouldn't cache everything in memory, and you would probably be right. That's the way the current codebase works though, and mucking with memory settings lets me keep it running while I have a look at the base problem.