how are you guys? Thank you in advance!
You may skip the following (context of the problem):
I have a lot of files which are data points of a Nasa satellite
(MERRA-2) of different cities. When I batch download them
using wget, they all come with a fixed name, and change the extension
to .1 .2 .3 […]. If I try to force rename it inside the wget command
line (let’s say, to the name of the city it corresponds to), it somehow
corrupts the file/data structure (it is a .csv)Because of that, I’m just downloading the default way and batch
renaming it afterward using Bulk Rename Utility (changing the .1 .2 .3
for .csv and having a prefix going from 1 to x)Now I have to rename each file to its correspondent city and year.
I’ve written a simple JS to generate the ren command prompt to do so
when I generate the wget download links, and it was working fine in
the tests.My problem is that each file, from the total of 2961, has 1.6Mb and
takes like 55 minutes to download (downloads at 500 bytes per second),
so I have to let it download overnight. The next day I’ve come to see
that the download was done, but only 100ish of all 141 cities have
been downloaded. That is happening day after day, and the problem
seems to be on the Nasa server. Other partners in this project say
they’re also seeing those speeds… from 500bytes to 2kbps…Having it fail the download of random cities renders my ren script
useless.Now I have a bunch of .csv files that have the same name (only
differing by a prefix from 1 to 100), that I need to rename with its
city name. The only way of knowing which city a given file corresponds
to is to open it and look for the latitude data, and referring to
another table I have here.
I need a script that iterates on all the .txt files inside a folder, finds, let’s say, the latitude “-3.58035”, and renames that file for “CITYNAME”, changing its format from .txt to .csv, or at least maintaining it as .txt.
With this script in hand, I can write some more JS to generate scripts to change the names of the files based on each of the 141 different latitudes I have.
Here is a sample of the file content:
time,latitude[unit=”degrees_north”],longitude[unit=”degrees_east”],BCSMASS[unit=”kg m-3″],DUSMASS[unit=”kg m-3″],DUSMASS25[unit=”kg m-3″],OCSMASS[unit=”kg m-3″],SO4SMASS[unit=”kg m-3″],SSSMASS[unit=”kg m-3″],SSSMASS25[unit=”kg m-3″]
2000-01-01T00:30:00Z,-3.58035,-59.1315,2.800106813083403E-10,3.508830559439957E-9,9.654286259319633E-10,3.5261109587736428E-9,2.061603110448118E-10,3.2207483902091383E-10,8.28777324546337E-11
In this case, “-3.58035” is the latitude.
Can someone help me with that? I would really appreciate that!
