Bad wireframe projection using dlib landmarks

[cfoch]

Hello. I will start the description of the issue by showing an image

I am using dlib to generate the landmarks, and taking as a base the example located at examples/fit-model.cpp. Unfortunately the result is obviously unexpected. I really have no idea what's going on. I do not think that is a problem of index ordering. I wonder if you may have any idea about this.

This is the code and the only main thing that I change is the way I read the landmarks that is from lines 187 to 193.

I also have to say that the original example with the landmarks located at image_0010.pts works well.

[cfoch]

I confirm that the problem is caused by building it in release mode with

set(CMAKE_BUILD_TYPE Release)

I at least need the Release mode to use it with dlib, otherwise dlib is too slow without the release mode. Why may this occur when I build this on release mode?

[cfoch]

A guy from the ##c++ IRC channel suggest to compile with -DCMAKE_C_FLAGS="-Wall -Wextra" and fix all the warnings, since there may be an undefined behavior.

[patrikhuber]

Hi,

So you're saying it works correctly in Debug mode, and in Release mode it produces the picture above? How about in RelWithDebInfo mode?
Your code how you use the dlib landmarks looks okay on a very quick glance, given that you're using the 68-point dlib detector and they're in the ibug-68 order?
It actually looks like you may have some mistake with the landmark ids/order, if you look at this figure https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/, and compare with the red numbers on your screenshot, they're off-by-one, for example the left contour top point is 16 in your pic but 17 in the ibug scheme.
This is very likely a user error on your side, I suggest you check everything and debug/step through your code.

[cfoch]

I will double check. But this is not the problem in the index order. I just compile the same code from the examples with "set(CMAKE_BUILD_TYPE Release)" and I get this problem. I remove the "set(CMAKE_BUILD_TYPE Release)" and it renders as expected. Just my Makefile is a bit different from yours.

Here is my CMakeLists.txt somewhat hard-coded because I don't know the best way to include eos yet.

[cfoch]

The problem is because I am compiling with

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_BUILD_TYPE Release)

So the combination of using -std=c++14 and Release mode is what leads to this problem. Just add those lines to the CMakeLists.txt from the examples and you should be able to reproduce the error.

[cfoch]

The reason why I compile with C++14 standard is because otherwise I get this error when I compile the code with eos+dlib: https://gist.github.com/cfoch/64b116cc1a2449051051ff14a9702daf

[patrikhuber]

Can you post your platform, exact compiler version and command-line output of the compiler call when compiling & linking the fit-model target? If you use a Makefile generator you should be able to do the last with make VERBOSE=1.

[cfoch]

Sorry for the delay...

[cfoch@localhost bin]$ uname -rio
4.18.16-300.fc29.x86_64 x86_64 GNU/Linux

[cfoch@localhost bin]$ cat /etc/redhat-release 
Fedora release 29 (Twenty Nine)

gcc/g++ version

[cfoch@localhost bin]$ g++ --version
g++ (GCC) 8.2.1 20180801 (Red Hat 8.2.1-2)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[cfoch@localhost bin]$ gcc --version
gcc (GCC) 8.2.1 20180801 (Red Hat 8.2.1-2)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Command line output when compiling fit-model with cmake (with set(CMAKE_CXX_STANDARD 14), set(CMAKE_CXX_STANDARD_REQUIRED ON) and set(CMAKE_BUILD_TYPE Release)):

[ 12%] Building CXX object examples/CMakeFiles/fit-model.dir/fit-model.cpp.o
cd /home/cfoch/dev/checkout/misc/eos/build/examples && /usr/bin/c++   -I/home/cfoch/dev/checkout/misc/eos/include -I/home/cfoch/dev/checkout/misc/eos/3rdparty/cereal/include -I/home/cfoch/dev/checkout/misc/eos/3rdparty/eigen -I/home/cfoch/dev/checkout/misc/eos/3rdparty/glm -I/home/cfoch/dev/checkout/misc/eos/3rdparty/nanoflann/include -I/home/cfoch/dev/checkout/misc/eos/3rdparty/eigen3-nnls/src -I/home/cfoch/dev/checkout/misc/eos/3rdparty/toml11 -isystem /usr/include/opencv  -O2 -DNDEBUG   -std=c++14 -o CMakeFiles/fit-model.dir/fit-model.cpp.o -c /home/cfoch/dev/checkout/misc/eos/examples/fit-model.cpp
[ 25%] Linking CXX executable fit-model
cd /home/cfoch/dev/checkout/misc/eos/build/examples && /usr/bin/cmake -E cmake_link_script CMakeFiles/fit-model.dir/link.txt --verbose=1
/usr/bin/c++  -O2 -DNDEBUG   CMakeFiles/fit-model.dir/fit-model.cpp.o  -o fit-model /usr/lib64/libopencv_imgcodecs.so.3.4.1 /usr/lib64/libboost_system.so /usr/lib64/libboost_filesystem.so /usr/lib64/libboost_program_options.so -pthread /usr/lib64/libopencv_imgproc.so.3.4.1 /usr/lib64/libopencv_core.so.3.4.1 
make[2]: Leaving directory '/home/cfoch/dev/checkout/misc/eos/build'
[ 25%] Built target fit-model
make -f examples/CMakeFiles/fit-model-multi.dir/build.make examples/CMakeFiles/fit-model-multi.dir/depend
make[2]: Entering directory '/home/cfoch/dev/checkout/misc/eos/build'
cd /home/cfoch/dev/checkout/misc/eos/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/cfoch/dev/checkout/misc/eos /home/cfoch/dev/checkout/misc/eos/examples /home/cfoch/dev/checkout/misc/eos/build /home/cfoch/dev/checkout/misc/eos/build/examples /home/cfoch/dev/checkout/misc/eos/build/examples/CMakeFiles/fit-model-multi.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/cfoch/dev/checkout/misc/eos/build'
make -f examples/CMakeFiles/fit-model-multi.dir/build.make examples/CMakeFiles/fit-model-multi.dir/build
make[2]: Entering directory '/home/cfoch/dev/checkout/misc/eos/build'

Compiling as the the project is by default:

[ 12%] Building CXX object examples/CMakeFiles/fit-model.dir/fit-model.cpp.o
cd /home/cfoch/dev/checkout/misc/eos/build/examples && /usr/bin/c++   -I/home/cfoch/dev/checkout/misc/eos/include -I/home/cfoch/dev/checkout/misc/eos/3rdparty/cereal/include -I/home/cfoch/dev/checkout/misc/eos/3rdparty/eigen -I/home/cfoch/dev/checkout/misc/eos/3rdparty/glm -I/home/cfoch/dev/checkout/misc/eos/3rdparty/nanoflann/include -I/home/cfoch/dev/checkout/misc/eos/3rdparty/eigen3-nnls/src -I/home/cfoch/dev/checkout/misc/eos/3rdparty/toml11 -isystem /usr/include/opencv  -std=c++17 -o CMakeFiles/fit-model.dir/fit-model.cpp.o -c /home/cfoch/dev/checkout/misc/eos/examples/fit-model.cpp
[ 25%] Linking CXX executable fit-model
cd /home/cfoch/dev/checkout/misc/eos/build/examples && /usr/bin/cmake -E cmake_link_script CMakeFiles/fit-model.dir/link.txt --verbose=1
/usr/bin/c++     CMakeFiles/fit-model.dir/fit-model.cpp.o  -o fit-model /usr/lib64/libopencv_imgcodecs.so.3.4.1 /usr/lib64/libboost_system.so /usr/lib64/libboost_filesystem.so /usr/lib64/libboost_program_options.so -pthread /usr/lib64/libopencv_imgproc.so.3.4.1 /usr/lib64/libopencv_core.so.3.4.1 
make[2]: Leaving directory '/home/cfoch/dev/checkout/misc/eos/build'
[ 25%] Built target fit-model
make -f examples/CMakeFiles/fit-model-multi.dir/build.make examples/CMakeFiles/fit-model-multi.dir/depend
make[2]: Entering directory '/home/cfoch/dev/checkout/misc/eos/build'
cd /home/cfoch/dev/checkout/misc/eos/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/cfoch/dev/checkout/misc/eos /home/cfoch/dev/checkout/misc/eos/examples /home/cfoch/dev/checkout/misc/eos/build /home/cfoch/dev/checkout/misc/eos/build/examples /home/cfoch/dev/checkout/misc/eos/build/examples/CMakeFiles/fit-model-multi.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/cfoch/dev/checkout/misc/eos/build'
make -f examples/CMakeFiles/fit-model-multi.dir/build.make examples/CMakeFiles/fit-model-multi.dir/build
make[2]: Entering directory '/home/cfoch/dev/checkout/misc/eos/build'

[cfoch]

Thank you. I solved it basically after checking the command line output using VERBOSE=1 as you suggested.

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_BUILD_TYPE Release)

made the trick! 💯

[patrikhuber]

So what was the problem? Does it work in all configurations now?

[cfoch]

It is enough to me having this "just working". However, it does not work when building it on the C++14 standard. But since the description of this project states "A lightweight 3D Morphable Face Model fitting library in modern C++11/14", I think that this should work compiling it in the C++14 standard.

[patrikhuber]

I see. It should definitely work in all configurations and on all compilers, and it does so on all the ones that I've tested. If I'll ever find some time, I might test on your system/compiler configuration. I still think it's most likely that this is a problem on your system, and also this is the first report of such kind. I'll leave the issue open for now.