Toems-API using all available RAM

aid

I'm having an issue where IIS for Toems-API will, as shown, use all available system memory, usually followed by the database crashing because it's out of memory. This is repeating every hour or so. Is there a config issue that can cause this, or are more logs needed to diagnose?

aid

It appears to be an issue with the Data Cleanup task. If I click Run Now, the process immediately starts consuming more memory until it crashes.

theopenem_admin

difficult to say what's happening, if you put your application in debug mode and post the log, then I can see exactly what part of the cleanup it's hanging on, and look for possible culprits.

aid

I'll work on that, where is the option for debug mode?

Another possibly related piece of info is that there is one piece of software taking up 70k+ entries in the software list; it looks like it's creating a new entry every inventory instead of merging with the existing line in the database. This is causing weekly software inventories to bring the server up to 100% CPU usage for hours at a time. I'll attach some screenshots of what I've found with that in case it's related.

SQL processes during the inventory and 100% usage on the database server:

The software list:

The software in the database:

(The 55,952 number is way off; there are 86,697 rows in that table and 71,708 of them are that 3M driver.)

Could I safely remove unreferenced instances of this from the software_inventory table? What tables would I need to check to do so?

aid

It appears these are two separate issues. The RAM usage during data cleanup is being caused while running this query:

 SELECT
`Extent1`.`computer_process_history_id`, 
`Extent1`.`computer_id`, 
`Extent1`.`process_id`, 
`Extent1`.`username`, 
`Extent1`.`start_time_utc`, 
`Extent1`.`close_time_utc`
FROM `computer_process_history` AS `Extent1`
 WHERE (`Extent1`.`start_time_utc` < '2024-12-17 13:05:46.557540') AND (`Extent1`.`start_time_utc` > '1996-01-29 13:05:46.272868')

This appears to be related: https://stackoverflow.com/questions/21568479/how-can-i-delete-1-000-rows-with-ef6
Since EF Core 7.0+ has new commands to fix this, hopefully it won't be an issue in Theopenem 2.0. Until then, I'll just do a process history cleanup manually every now and then if necessary.

As far as the software inventory issue goes, none of the instances of that 3M driver were referenced in computer_software. I took a backup of the table and deleted all of them, and we'll see if bulk inventories no longer lock up the server for multiple hours.

aid

The second issue was being caused by a Version string that was way longer than it should be, as it includes the name of the application for some reason.

It's 47 characters long, which didn't fit in the varchar(45) software_inventory.version. I set the max length to 50 and the inventory is working properly now. I'd personally suggest a higher default limit on that to avoid this problem, or truncate the version before comparing it with the database, which is what lead to the 60k duplicate entries.