Compiling using native windows Visual Studio compiler

[Tripphippie]

I'm attempting to build Essentia using Visual studio and native windows tools. I'm not a linux guy and I don't want to setup cygwin and all that nonsense. Would someone be so kind as to send me a copy of essentia_algorithms_reg.cpp that I could use? I'm going to replace the python with something else and I need a template to know what to output. If anyone is interested in native windows builds I might be able to upstream my changes. Also what is up with the RogueVector class. The methods that access private members of std::vector won't compile but I think with swap semantics and the c++11 compiler features you shouldn't have to use the rogueVector at all.

[dbogdanov]

Sure, here is the file, and your contributions are more than welcome.

[Tripphippie]

Thanks so much. Any ideas on what to do with RogueVector? It's basically a hack and it won't compile in this version of VCC. I'm almost convinced I can remove it completely but I need to run some tests to be sure.

[dbogdanov]

Try changing back to the the old code of RogueVector for WIN32 which is now commented.

[q-depot]

I've been trying to compile with VS for months, still can't get it to work, for the RogueVector I've replace these functions:

template <typename T> void RogueVector<T>::setData(T* data) { T* this_data = this->data(); this_data = data;

// this->_M_impl._M_start = data; }

template <typename T> void RogueVector<T>::setSize(size_t size) { this->resize( size ); this->shrink_to_fit();

// this->_M_impl._M_finish = this->_M_impl._M_start + size; // this->_M_impl._M_end_of_storage = this->_M_impl._M_start + size; }

@dbogdanov is there any chance somebody can look at a proper VS solution? seems many people would like to use it on windows and to be honest I don't know anybody using codeblocks.

thanks

[Tripphippie]

@q-depot Does that work? I may give your implementation a go. @dbogdanov That was the first thing I tried to do. The implementation has changed and that code doesn't compile anymore in VS. If I understood what the RogueVector was trying to accomplish then I think I can come up with a good implementation for Visual Studio. I think the swap constructor does what RogueVector is trying to do which is avoid a copy.

[q-depot]

I've managed to compile everything in VS and build a static library, but the library itself didn't work so I couldn't test that implementation. Besides the roguevector there are many other issues you need to go through, namespace, includes etc. I'm about to have another go so I'll keep you posted, but I think I'm giving up with the static library, the one I've compiled was about 1gb( osx static lib is 40mb), not sure why it gets so big.

[Tripphippie]

I have a static lib working right now and I've managed to link it to a test application but I don't have roguevector implemented. I ran my test app and it stopped on RogueVector because I put a _DebugBreak() stub in there to remind me to fix it. I'm not sure why the lib would be so big. mine is also 1.2gb. I had to pull a fresh build of ffmpeg to get stuff to link. I'm going to try your version of RogueVector next as I've spent most of today resolving libav dependencies.

[q-depot]

IMO the 1.2gb lib is not usable, I also reckon it should be compile without the dependencies.

[Tripphippie]

I think it might have something to do with the dependencies why it is so large. I'm going to try to make it into a DLL after I get this static lib working. Then I might be able to see why it is so large. I think we might have a situation where the entire cygwin c-runtime is linked into this static lib. All the dependencies would need to be recompiled under windows to correct the problem. Keep in mind this is under a debug build so it may shrink when all the debug junk is removed.

[q-depot]

the static library in release was about 700-800Mb so still huge.

Today I've made some progress, as I said I gave up(for now) with the static library, instead I've compiled the source straight in my app, I've stripped out most of the library and all the dependencies. I had to fix a few issues mostly with the cmath library, and I uncommented the roguevector implementation which seems to be working. The current implementation is not fully working but at least I can build and compute(badly) the mfcc.

[MGarvinNYC]

I'm glad to hear that there's an effort to build a Windows/VS version!

Re the size (800MB?!): There is supposedly a way to cross-compile a Windows lib from Linux. I don't think that the library will do much good in itself, as it's not VS-compatible, and supposedly has no Python bindings. But perhaps it would give an indication of size to expect in a reasonable VS build.

[dbogdanov]

800mb is so huge! :open_mouth: The cross-compiled library using mingw-w64 on Linux should be no more than 20mb.

[dbogdanov]

@q-depot Your fix for RogueVector using data(), resize() and shrink_to_fit() looks dangerous. If the size of the vector was smaller before setSize, I would expect resize() to mess up with the data in the data() pointer.

[q-depot]

I'm using the commented code you left there. It seems fine

[q-depot]

I've found another issue while building in debug, the static free method in algorithfactory.h seems in conflict with the reserved one, I had to rename it to something like this:

static void freeAlgo(BaseAlgorithm* algo) { delete algo; }

it would be good to fix it in the master branch.

[Tripphippie]

Ok so a little update on this. I have a static lib working mostly. I have a dll that is much much smaller (7mb) but is missing a bunch of ESSENTIA_API export tags, it will grow as I add missing things. I can't get taglib to work at all since the dll in the package folder is for gcc or something. So MetadataReader is missing from the lib. I'm not actually sure at this time how I got taglib working in the static version of the library, strange. I'm currently decorating missing functions but it is impossible to know which ones are missing until I link with an test exe so I may not find them all.

EDIT: Oh man! I think I just stumbled across some dark hole of dllimport and headers and inlining and dlls. I may have to refactor this whole thing a little. I get link errors on global variables and statics.

[q-depot]

what errors are you getting? I had similar problems with kissfft

[Tripphippie]

Taglib won't link because the lib file is missing and I'm too lazy to compile the whole thing. Even if it existed linker would be looking for the wrong name mangling scheme as the DLL is full of gcc names. I didn't have problems with kiss as the source is there. I just pulled the source into my project. I'm having problems getting certain symbols to export in the DLL. I'm going to have to get all the test programs to compile and link and then maybe I have a chance of getting a full complete DLL. Problem is that gcc exports most things by default while MSVC requires templates to be instantiated. I had to add some template instance declarations to force the template instance to be created and exported. Now I'm stuck on getting statics and globals to export properly. Here's my current error list.

2>BasicExtractor.obj : error LNK2001: unresolved external symbol "int essentia::activatedDebugLevels" (?activatedDebugLevels@essentia@@3HA)
2>BasicExtractor.obj : error LNK2001: unresolved external symbol "int essentia::debugIndentLevel" (?debugIndentLevel@essentia@@3HA)
2>BasicExtractor.obj : error LNK2001: unresolved external symbol "class essentia::Logger essentia::loggerInstance" (?loggerInstance@essentia@@3VLogger@1@A)
2>BasicExtractor.obj : error LNK2001: unresolved external symbol "public: static class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const essentia::standard::Algorithm::processingMode" (?processingMode@Algorithm@standard@essentia@@2V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@B)
2>BasicExtractor.obj : error LNK2001: unresolved external symbol "private: static class essentia::EssentiaFactory<class essentia::standard::Algorithm> * essentia::EssentiaFactory<class essentia::standard::Algorithm>::_instance" (?_instance@?$EssentiaFactory@VAlgorithm@standard@essentia@@@essentia@@0PAV12@A)

[MGarvinNYC]

I'm curious about how this is going. Stalled for the holidays?

[Essentia]

Hello,

Somehow my company email and name (ESSENTIA) got linked to this post .... As far as I could read on the post, it's related music files?

---

Gabor Simonsich

Desarrollador de Negocios Business Developer Essentia

Cel: +598 95 120021 [email protected] www.essentia.com.uy

On Wed, Dec 30, 2015 at 4:22 AM, MGarvinNYC [email protected] wrote:

I'm curious about how this is going. Stalled for the holidays?

— Reply to this email directly or view it on GitHub https://github.com/MTG/essentia/issues/330#issuecomment-167950439.

[dbogdanov]

@Essentia Yes it is. You got referred accidentally, sorry for that.

[dbogdanov]

I've found this for building taglib for Visual Studio: http://stackoverflow.com/a/5365962

[Tripphippie]

This has come to a halt for the time being. Holidays is over and the skunk-works time I had at work has expired with it. If I disable the debug macros I can get it down to 1 variable not linking properly. I'm not sure how to fix this problem.I think this link is relevant. http://stackoverflow.com/questions/17614172/c-template-singletons-in-a-dll So I think I need to move some stuff around in the headers and have templates in cpp files which is weird. last error.... 2>BasicExtractor.obj : error LNK2001: unresolved external symbol "private: static class essentia::EssentiaFactory<class essentia::standard::Algorithm> * essentia::EssentiaFactory<class essentia::standard::Algorithm>::_instance" (?_instance@?$EssentiaFactory@VAlgorithm@standard@essentia@@@essentia@@0PAV12@A)

[MGarvinNYC]

Sorry to hear that the real world is intruding! :-) It sounds like you're so close, too.

Perhaps Dmitri or others could shed some light on that last problem?

[Tripphippie]

I cracked some sort of solution but I don't like it. None of my other solutions have worked except this. If I compile the DLL with #include "algorithmfactory_impl.h" and then comment that line out when I include it in my executable it works. #ifdef'ing the line out doesn't work for some reason. neither does moving the code from algorithmfactory_impl.h to a cpp file and only including it in the dll project. I think I might try moving the code to a "algorithmfactory_private.h" and making a copy to use when importing the DLL. Basically the templated function is expanding in both compilation units and then failing to link because it is in a different compilation unit, it needs to only include the basic interface and not any of the template function bodies. Windows DLLs differ from SOs on linux in that they don't invoke the linker when binding. you can end up with the same templated classes in both objs and statics become a problem.

edit: nevermind. I'm still not sure why this works but I moved the template instantiation to the header with the extern keyword and it works without commenting out lines selectively. Now to actually try to use some algorithms and probably sort out some more issues since this is like a basic basic link test that is working.

double edit: Seems all the win32 3rd party dlls don't match the libs that are right next to them. Need to get win32 libs for ffmpeg.

[mathieu-b]

Hi I am also trying to have Essentia up and running under Visual Studio 2015. Any updates about how to possibly implement RogueVector (or an equivalent) under VisualStudio? @wackou / @Tripphippie / @dbogdanov ?

Are there any resources detailing the RogueVector specifications and usage?

Thanks

[wackou]

the RogueVector initially was a hack in C++98 to support non-owning vector and allow a nicer api than passing (float* array, int size) as arguments to all functions.

A better solution (since C++11) would be to use a std::array (http://en.cppreference.com/w/cpp/container/array) which has the efficiency of passing raw pointers around but has a nice object-oriented api. That involves changing a bit of code, but it probably would be worth it in the long run. Alternatively, one could define an essentia::Array type, use it everywhere in the api, and typedef it to a given type (std::array for instance)

For a quick solution, though, I maybe looking at the source code of roguevector (https://github.com/MTG/essentia/blob/master/src/essentia/roguevector.h#L77) and see how it could be adapted to the (I imagine) new libstdc++ from the latest Visual Studio would be faster.

[mathieu-b]

Ok, thanks for the clarification and suggestions @wackou :)

[q-depot]

is there any update on this? I've managed to compile in VisualStudio a while ago, I had to do many tweaks and hacks. Now I want to update to the latest and I basically have to start from scratch. has the code been updated to c++11?

[dbogdanov]

Check the python_win branch, there's some work done to be able to compile with VS. See the build instructions here. We have prebuilt 32 bit dependencies, but we are missing some of the 64bit. We'll add a download link if you need those.