| themaManager http://manager.themasoftware.com/forum/ |
|
| Large number of posts replacements from text file. http://manager.themasoftware.com/forum/viewtopic.php?f=4&t=7643 |
Page 1 of 1 |
| Author: | Crackpot [ January 16th, 2024, 5:20 pm ] |
| Post subject: | Large number of posts replacements from text file. |
I tried to replace around 13K post from text file, but memory use jumped to 50gb+ while editing the thread it was still same memory. i abort the edit. restart the TM and then edit. it started to work fine.  also last time it requested while replacing text file, add a the text file load. but after text file load done TM checking with subject for replacement i guess. can we add something more see the processing, like X% checking done or something. |
|
| Author: | Freddy [ January 16th, 2024, 6:55 pm ] |
| Post subject: | Re: Large number of replacement from text file |
Text files need to be read to memory, so it might not be possible to avoid that jump. If it's really a lot of files just do it in batches, like divide them in two or three parts and replace only half / third at first. You can see background progress in "STATUS" -> "EDITS" -> at the bottom "Background log" table -> it shows task progress with how many websites to check are still left and posts counter going down for each site it checks. |
|
| Author: | Crackpot [ January 17th, 2024, 12:44 pm ] |
| Post subject: | Re: Large number of posts replacements from text file. |
when we load text file, memory jump sky rocket (more than 10K text file). but once load is done it takes some time to come down a bit. like 23gb ram. once click submit to replace. it jump back to 50gb+ memory. i saw it same for 5~6 hour and even after 4K post is edited memory never come down. it should come down to memory a lot like 2~3gb. i know that because when i saw memory is too high. i abort all the edit. restart the TM and retry and then i saw even with 14K edit post running TM using only 800mb memory here total post is editing 15140 https://i.imgur.com/yiy0lbv.png |
|
| Author: | Freddy [ January 17th, 2024, 1:19 pm ] |
| Post subject: | Re: Large number of posts replacements from text file. |
Will take a look more. |
|
| Author: | Freddy [ February 12th, 2024, 7:59 pm ] |
| Post subject: | Re: Large number of posts replacements from text file. |
3.89: * Added option in "Settings" -> "Edits" -> "Automatically remove edit statuses older than (in days)". Which should help a bit to clean-up old statuses which are not needed (they do take quite a lot of space and should be removed). And made one small other improvement, but not sure how much it will help. Might still need to take a look more. Edit: 3.90 * Added option in "Settings" -> "Edits" -> "Automatically remove success edit statuses after editing". * Improved saving on edit statuses. |
|
| Page 1 of 1 | All times are UTC |
| Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |
|