![]() ![]() I don't think that splitting this work onto multiple threads is going to provide much improvement, and may in fact make the problem worse by consuming even more memory. Do you know of a different approach to split at comas and replace the double quotes in each CSV line ? Would StringBuilder be of any healp here ? What about StringTokenizer ? How could I reduce the amount of heap memory used in the process ? Is the multithread implementation with Callable correct ? How could I improve the speed of the CSV reading ? Other than that the api is running out of heap memory when running on the server, I know that a solution would be to enhance the amount of available memory but I suspect that the replace() and split() operations on strings made in the Callable(s) are responsible for consuming a large amout of heap memory. To improve speed processing, I tried to implement multithreading with Callable(s) but I am not familiar with that kind of concept, so the implementation might be wrong. The code works fine on my local machine but it is very slow : it takes about 20 seconds to process 450 columns and 40 000 lines. I am not guaranteed to have the same header between files (each file can have a completly different header than another), so I have no way to create a dedicated class which would provide mapping with the CSV headers.Ĭurrently the api controller is calling a csv service which reads the CSV data using a BufferReader. It has to read big CSV files which will contain more than 500 columns and 2.5 millions lines each. I am currently working on a spring based API which has to transform csv data and to expose them as json. And has been a life saver in terms of time spent looking through my vault. For me this has reduced my vault entirely to 146 pieces of Legendary Armor across ALL my characters. Max Power will need to be changed manually every season. This allows you to keep junk gear to infuse into you gear that you want to be max power. The last part goes through untagged armor and marks any that is currently at max power gets marked as 'infuse'. If that happened to be in my vault that gear would be marked as 'favorite'. An example for me would be this for my gloves. This allows you to explictly put any of YOUR god rolls. The script will look for any armor that matches ALL of the defined perks. Perk Sets are sets of perks that YOU define. ![]() The second part then goes through and finds any defined Perk Sets and marks them as 'favorite'. Those weights are free for you to customize on what perks you prefer and the script will still keep at least one of every perk you define in the CSV. The weight is a nice little feature I added to allow it to further optimize gear choices, so that it would pick per say over. Within that CSV you define the name of the perk, the minimum to keep (Currently hard coded to 1 since a recent update required a change in how to implement it), and the weight of the perk to YOU. These are defined in the RecommendedPerks.csv. First the script simply looks at what YOU define the recommended perks are and marks items found as 'keep'. ![]() The tags differ depending how the gear is matched. This script uses tags from Destiny Item Manager (DIM) to mark what gear to keep. This was meant to help introduce me to python and yes my code may be a bit sloppy due to it. This started in powershell and later moved to python. I decided to make my own script to manage it. Over the past Season of the Drifter with all the new armor added I was looking for a way to easily optimize my inventory. Is your vault filled to the brim with armor? Looking for a way to quickly sort through it all? Look no further because this is the solution for you!. However Destiny2ItemManager build file is not available. Destiny2ItemManager has no bugs, it has no vulnerabilities and it has low support. Destiny2ItemManager is a Python library typically used in Utilities, CSV Processing applications. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |