seabird
seabird copied to clipboard
castelao
Read .hex files
Improve seabrid to be able to read directly the binaries .hex files. I believe it requires to combine information from .hdr
See https://github.com/jklymak/SeaCat for a matlab version of this. I'll probably translate to python soonish and will link to here.
https://github.com/jklymak/SeaCat now has the python code.
Its really just for our seacat. Not sure what other file formats there are, so may require more work to make general. OTOH most folks should be able to follow it and get something useful.
Thank you very much @jklymak . I'll work on that in the first chance.
Whenever I do it, would you mind if I copied one of your example files to include in my validation tests?
I would love this function. I am trying to systematically pull all the hex metadata from the files on our server, so that people can find unprocessed data by location and date.
Thanks, @jklymak. As I recall, I had a hard time getting https://github.com/jklymak/SeaCat to work for my data, but that was a couple of months ago, so I need to go back and note specifically what the problem was.
Its not general at all, so you'd need to adopt for different seabirds. But I don't have access to a bunch of CTD files, so haven't generalized it. Someone who works at WHOI may have access to more CTDs ;-)
Ha! Yes, I do have many files, from several different vessels over a long period of time. I'm just not much of a python programmer. I will certainly give it a shot at some point. Let me know if you need files. They are web accessible, but I can pull some together in an easier format.
It'd be useful that this repository had a bunch of example files for testing against...
Hi everyone,
@jklymak thanks again for sharing your code. I'm sorry that I haven't the chance yet to work on that. I'll figure out some time to work on that.
@atmickle thanks for the interest, that definitely stimulates me to figure time to work on this. How could we get some of your sample files? Are they publically available?
Thank you!
I'm happy to help (given time constraints) with integrating a more general solution. @atmickle if there were a good suite of sample hex files from diff instruments, that would be a great start.
@castelao @jklymak My use case is pretty limited--I'm just trying to pull basic metadata from the raw files, since not all the ships send along processed data in their packages--so I wouldn't ask you guys to spend any significant time on it, if others aren't looking for that. I can work around it.
But--It is literally my job to provide public data to those in need. Here are some select sets of .hex files, from 4 different vessels. I think all the metadata is apparent, but let me know if it isn't. All the data is list here by cruise and is also available from the R2R catalog. I can also provide FTP access to our raw data server upon request.
R2R also publishes their data as Linked Data, which can be queried or browsed. I don't think there's currently an easy way to get all CTD data without clicking through cruise level info, which is why my researcher asked me to work on this listing in the first place.
Thank you both for your time!
I'm happy to help (given time constraints) with integrating a more general solution.
@jklymak , that's great! The same here.
@atmickle , thanks for the links, those files will be perfect for this. Which metadata data would you like from the files?
It should be easy with @jklymak 's code and @atmickle 's files. I'll start something over the weekend and get back to you.
Thank you.
Geez. I'm sorry I didn't see that last week. I'm glad they are helpful! I'm just looking at the lat, long, time/date for searching purposes. Anything else I can pull is a bonus.