artoolkitx
Added "Buffer" type video module for uploading custom rgb images..
That is definitely interesting. Could you please have a look at the CI build issues and let me know if you can make sense of the error message and potentially fix them.
Cheers
Still failing
FAILED: /opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=x86_64-none-linux-android --gcc-toolchain=/opt/android-ndk/toolchains/x86_64-4.9/prebuilt/linux-x86_64 --sysroot=/opt/android-ndk/sysroot -DARVIDEO_INPUT_ANDROID -DARVIDEO_INPUT_BUFFER -DARVIDEO_INPUT_DEFAULT_ANDROID -DARVIDEO_INPUT_DUMMY -DARVIDEO_INPUT_IMAGE -DARX_EXPORTS=1 -DARX_TARGET_PLATFORM_ANDROID=1 -IARX/AR/include -I../../../../../../../ARX/ARVideo/include -I../../../../../../../depends/android/include -I../../../../../../../ARX/ARVideo/../AR/include -I../../../../../../../ARX/ARVideo/../ARUtil/include -isystem /opt/android-ndk/sysroot/usr/include/x86_64-linux-android -D__ANDROID_API__=21 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -O2 -DNDEBUG -fPIC -MD -MT ARX/ARVideo/CMakeFiles/ARVideo.dir/video2.c.o -MF ARX/ARVideo/CMakeFiles/ARVideo.dir/video2.c.o.d -o ARX/ARVideo/CMakeFiles/ARVideo.dir/video2.c.o -c /bitrise/src/Source/ARX/ARVideo/video2.c
In file included from /bitrise/src/Source/ARX/ARVideo/video2.c:78:
/bitrise/src/Source/ARX/ARVideo/Buffer/videoBuffer.h:66:2: error: unknown type name 'BYTE'
BYTE* userBufferPointer;
^
1 error generated.
ninja: build stopped: subcommand failed.
Fixed an issue, but still not sure why LInux and iOS won't build. Windows and Android are fine. Can you post the logs please? I can't see them through the bitrise app.
@XrizZ on Linux: ```CMake Error: The current CMakeCache.txt directory /bitrise/src/Source/CMakeCache.txt is different than the directory c:/Users/Chris/Desktop/artoolkitX_mine/Source where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
might be an absolute path somewhere
macos is the same ```CMake Error: The current CMakeCache.txt directory /Users/vagrant/git/Source/CMakeCache.txt is different than the directory c:/Users/Chris/Desktop/artoolkitX_mine/Source where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
@ThorstenBux thanks, the issue was just the CMakeCache.txt being in the repository, which it shouldnt. Whats the last remaining error for macos?
cd /Users/vagrant/git/Source
export LANG=en_US.US-ASCII
export PATH="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin:/Users/vagrant/.bitrise/tools:/usr/local/bin:/usr/local/sbin:~/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/opt/go/libexec/bin:/Users/vagrant/go/bin:/Users/vagrant/bitrise/tools/cmd-bridge/bin/osx"
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x c -arch arm64 -fmessage-length=0 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit=0 -Wno-trigraphs -fpascal-strings -O3 -Wno-missing-field-initializers -Wno-missing-prototypes -Wno-return-type -Wno-missing-braces -Wparentheses -Wswitch -Wno-unused-function -Wno-unused-label -Wno-unused-parameter -Wno-unused-variable -Wunused-value -Wno-empty-body -Wno-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wno-constant-conversion -Wno-int-conversion -Wno-bool-conversion -Wno-enum-conversion -Wno-float-conversion -Wno-non-literal-null-conversion -Wno-objc-literal-conversion -Wshorten-64-to-32 -Wpointer-sign -Wno-newline-eof -DCMAKE_INTDIR=\"Release-iphoneos\" -DARX_TARGET_PLATFORM_IOS=1 -DARX_EXPORTS=1 -DARVIDEO_INPUT_DUMMY -DARVIDEO_INPUT_IMAGE -DARVIDEO_INPUT_BUFFER -DARVIDEO_INPUT_AVFOUNDATION -DARVIDEO_INPUT_DEFAULT_AVFOUNDATION -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.3.sdk -fstrict-aliasing -Wdeprecated-declarations -miphoneos-version-min=8.0 -Wno-sign-conversion -Wno-infinite-recursion -Wno-comma -Wno-block-capture-autoreleasing -Wno-strict-prototypes -I/Users/vagrant/git/Source/build-ios/ARX/ARVideo/Release-iphoneos/include -I/Users/vagrant/git/Source/build-ios/ARX/AR/include -I/Users/vagrant/git/Source/ARX/ARVideo/include -I/Users/vagrant/git/Source/depends/ios/include -I/Users/vagrant/git/Source/ARX/ARVideo/../AR/include -I/Users/vagrant/git/Source/ARX/ARVideo/../ARUtil/include -I/Users/vagrant/git/Source/build-ios/ARX/ARVideo/artoolkitX.build/Release-iphoneos/ARVideo.build/DerivedSources/arm64 -I/Users/vagrant/git/Source/build-ios/ARX/ARVideo/artoolkitX.build/Release-iphoneos/ARVideo.build/DerivedSources -Wmost -Wno-four-char-constants -Wno-unknown-pragmas -F/Users/vagrant/git/Source/build-ios/ARX/ARVideo/Release-iphoneos -integrated-as -integrated-as -DNDEBUG -MMD -MT dependencies -MF /Users/vagrant/git/Source/build-ios/ARX/ARVideo/artoolkitX.build/Release-iphoneos/ARVideo.build/Objects-normal/arm64/videoBuffer.d --serialize-diagnostics /Users/vagrant/git/Source/build-ios/ARX/ARVideo/artoolkitX.build/Release-iphoneos/ARVideo.build/Objects-normal/arm64/videoBuffer.dia -c /Users/vagrant/git/Source/ARX/ARVideo/Buffer/videoBuffer.c -o /Users/vagrant/git/Source/build-ios/ARX/ARVideo/artoolkitX.build/Release-iphoneos/ARVideo.build/Objects-normal/arm64/videoBuffer.o
/Users/vagrant/git/Source/ARX/ARVideo/Buffer/videoBuffer.c:220:2: error: implicit declaration of function 'arUtilTimeSinceEpoch' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
arUtilTimeSinceEpoch(&vid->buffer.time.sec, &vid->buffer.time.usec);
^```
Thanks @ThorstenBux - its fixed now! I'm hopeful that the feature can be merged into your master branch now. Cheers!
@XrizZ if you can spare some time could you write down some documentation on how to use your feature?
@philip-lamb Could you please review and merge if approved, tests are all good.
Sure thing @philip-lamb, I would recommend this as documentation:
The "Buffer" type video module offers the ability to feed in custom sources of frames to the ARToolkitX-AR Controller. The usage is similar to using the webcam type video module, however rather than calling the update() method on the AR controller without parameters and getting frames from the webccam, it is now possible to pass one or more buffers (in case of stereo video) along with the update function. The update function is now declared as:
/** * Performs tracking and updates all trackables. The latest frame from the current * video source is retrieved and analysed. Each trackable in the collection is updated with * new tracking information. The trackable info array is * iterated over, and detected trackables are matched up with those in the trackable collection. Each matched * trackable is updated with visibility and transformation information. Any trackables not detected are considered * not currently visible. * * leftBuffer and rightBuffer only needs to be passed in by reference if the video type is buffer, otherwise these * parameters don't need to be set at all or can be NULL, buffer size and type needs to correspond to the resolution * and pixelformat specified during initialization * * rightbuffer only needs to be set if stereo view is selected * * @return true if update completed successfully, false if an error occurred / bool update(AR2VideoBufferT leftBuffer = NULL, AR2VideoBufferT* rightBuffer = NULL);
For further explanations please see the example code below:
Initialization: ` // Initialise the ARController. m_arController = new ARController(); bool success = m_arController->initialiseBase(); if (!success) { CString error = "Error initialising ARController.\n"; m_arController = NULL; return false; }
// Add trackables. m_arMarkerCount = (sizeof(MARKERS) / sizeof(MARKERS[0])); m_arMarkerIDs = new unsigned int[m_arMarkerCount]; m_arMarkerModelIDs = new unsigned int[m_arMarkerCount];
for (int i = 0; i < m_arMarkerCount; i++) { std::string markerConfig = "single;" + resourcePath + '\' + MARKERS[i].name + ';' + std::to_string(MARKERS[i].height); m_arMarkerIDs[i] = m_arController->addTrackable(markerConfig); if (m_arMarkerIDs[i] == -1) { CString error = "Error adding marker."; return false; } } if (m_arController->getSquareTracker()) { m_arController->getSquareTracker()->setPatternDetectionMode(AR_TEMPLATE_MATCHING_MONO); m_arController->getSquareTracker()->setThresholdMode(AR_LABELING_THRESH_MODE_AUTO_BRACKETING); } else { CString error = "Cannot find square tracker module."; return false; }
CString vconf;
vconf = "-module=Buffer -format=RGB";
CString dim;
dim.Format(" -width=%d -height=%d", m_arWebcamWidth, m_arWebcamHeight);
vconf += dim;
CString log;
log.Format("vconf is '%s'.\n", vconf);
// Start tracking.
bool started = m_arController->startRunning(vconf, NULL, NULL, 0);
if (!started)
{
CString error = "Couldn't start AR tracking.";
return false;
}
m_isInitialized = true;
**Draw & Update**
if (!m_arController || !m_isInitialized)
return; //ERROR!
AR2VideoBufferT* buf = NULL;
if(!m_userVideoBuffer)
{
bool gotFrame = m_arController->capture();
if (!gotFrame)
{
return;
}
}
else
{
buf = new AR2VideoBufferT();
CString strFileName;
strFileName = "arTestImage.jpg";
//the buffer format needs to match pixel format and resolution as defined in the Init(..) function
buf->buff = LoadImageUsingOpenCV(strFileName)->data;
}
if (!m_arController->update(buf))
{
CString error = "Error in ARController::update().";
}
if (m_isFirstARFrame)
{
bool drawVideoInitSuccess = m_arController->drawVideoInit(0);
if (!drawVideoInitSuccess)
{
CString error = "Error in ARController::drawVideoInit().";
}
bool inited = m_arController->isInited();
if(!inited)
{
CString error = "Error ARController not initialized.";
}
int w = 0, h = 0;
AR_PIXEL_FORMAT pixF = AR_PIXEL_FORMAT::AR_PIXEL_FORMAT_BGRA;
m_arController->videoParameters(0, &w, &h, &pixF);
int width = GetDesiredWidth();
int height = GetDesiredHeight();
bool settingsSuccess = m_arController->drawVideoSettings(0, width, height, false, false, false, ARVideoView::HorizontalAlignment::H_ALIGN_CENTRE, ARVideoView::VerticalAlignment::V_ALIGN_CENTRE, ARVideoView::ScalingMode::SCALE_MODE_FIT, m_arViewport);
if (!settingsSuccess)
{
CString error = "Error in ARController::drawVideoSettings().";
}
SetViewport(m_arViewport);
ARdouble projectionARD[16];
m_arController->projectionMatrix(0, 10.0f, 10000.0f, projectionARD);
for (int i = 0; i < 16; i++)
m_gProjection[i] = (float)projectionARD[i];
SetCamera(m_gProjection, NULL);
for (int i = 0; i < m_arMarkerCount; i++)
{
m_arMarkerModelIDs[i] = DrawLoadModel();
}
m_isFirstARFrame = false;
}
// Clear the context.
glClearColor(0.0, 0.0, 0.0, 1.0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// Display the current video frame to the current OpenGL context.
glDisable(GL_LIGHTING);
glDisable(GL_BLEND);
if(!m_userVideoBuffer)
m_arController->drawVideo(0);
// Look for trackables, and draw on each found one.
for (int i = 0; i < m_arMarkerCount; i++)
{
// Find the trackable for the given trackable ID.
ARTrackable *marker = m_arController->findTrackable(m_arMarkerIDs[i]);
float view[16];
if (marker->visible)
{
for (int i = 0; i < 16; i++)
view[i] = (float)marker->transformationMatrix[i];
}
DrawSetModel(m_arMarkerModelIDs[i], marker->visible, view);
}
DrawARScene();
if(buf)
delete buf;
`
Hi,
First of all – thanks for this amazing work! I wonder if there was some news regarding this pull request.
I've been trying to make it work with 2D markers, but still no result ( zero detection ) @XrizZ Any chance we can be in touch so I can understand & help to make this feature happen ?
Any chance you can post / send a working example of your implementation ? I've been trying to send RGB buffer + 2D marker, with no luck. I wonder what is doing your LoadImageUsingOpenCV method for example.
Then, surely this is not the right place to go forward with this, any contact where I can reach you ? Many thanks !
@martial send me an email on my public GitHub address: [email protected]
I'll be able to send you a sample file back to your email next week. Unfortunately, I won't have access to the fiels sooner.. sorry.
Sorry that this contribution has sat unmerged. I will take a look at it in the next 24 hours.
Closing this now after all this time, as it has been made obsolete by #160 .