At the moment I am working with a program created by Mykhaylo Andriluka that is able to detect people in images in order to estimate their poses using a method called Pictorial Structures. The program is downloadable from his website but is not easy to compile and run. Here is what I did in order to get it to work so that it might help someone else too.

How to compile the program

At first, download the code and experiments from http://www.d2.mpi-inf.mpg.de/node/381. I am currently running Microsoft Windows, so I created a Mint Linux virtual machine using Oracle VM VirtualBox. A fresh install is always good to see what dependencies are needed. Following the README.txt accompanying the code I first installed QT, Boost, Matlab and Google Protocol Buffers. The PNG libraries, libboost-program-options-dev and libstdc++5 were also needed but not stated in the README.

sudo apt-get install libqt4-dev
sudo apt-get install libpng++-dev
sudo apt-get install libboost-program-options-dev
sudo apt-get install libstdc++5
sudo apt-get install libstdc++5-dev

Matlab version is 7 R14. Google Protocol Buffers version 2.3.0.

Following the README I continued to create symlinks to the library and include directories like this:

  ln -s /home/frank/andriluka/matlab/extern/include/ include_mat
  ln -s /home/frank/andriluka/matlab/bin/glnx86/ lib_mat
  ln -s /usr/local/include/google/protobuf/ include_pb
  ln -s /usr/local/lib/ lib_pb

The installed directory for the application is /home/frank/andriluka/partapp-r2 and these commands were run from there. Next step is to go into the src/libs and src/apps dir and run qmake; make. This was not working right away as there were some problems with the code. This is what I ran into and how I solved it:

Problem I:
../libKMA2/gauss_iir/../ImageContent/imageContent.h:25:17: fatal error: png.h:
No such file or directory

Solution:
sudo apt-get install libpng++-dev

Problem II:
../libAdaBoost/AdaBoost.h:128:20: error: ‘uint’ was not declared in this scope

Solution:
#include <sys/types.h>
In Adaboost.h:22 

Problem III:
../libMatlabIO/matlab_cell_io.hpp:65:17: error: ‘mwIndex’ was not declared in this scope
matlab_io.hpp:87:5: error: ‘mwSize’ was not declared in this scope

Solution (hack?):
In matlab file matrix.h
#define mwSize int
#define mwIndex int

Problem IV:
../libAnnotation/annorect.h: In member function ‘int AnnoRect::w() const’:
../libAnnotation/annorect.h:72:38: error: ‘abs’ was not declared in this scope

Solution:
In annorect.h:
#include <stdlib.h>

Problem V:
ImageContent/ImageContent.cpp:383:21: error: ‘atoi’ was not declared in this scope

Solution:
In imageContent.h:
#include <stdlib.h>

Problem VI:
kmaimagecontent.h:74:52: error: ‘strcpy’ was not declared in this scope

Solution:
kmaimagecontenmat.h
#include <string.h>

Probleem VII:
annotation.cpp:251:69: error: ‘stable_sort’ was not declared in this scope

Solution:
#include <algorithm>
In annotation.h 

Probleem VIII:
xmlhelpers.cpp:48:33: error: ‘atoi’ was not declared in this scope

Solution:
xmlhelpers.h:
#include <string.h>

Problem IX:
/usr/bin/ld: cannot find -lboost_program_options

Solution:
; sudo apt-get install libboost-program-options-dev

Problem X:
../../../lib/Release/libPartDetect.so: undefined reference to `mxCreateCellArray'
../../../lib/Release/libPartDetect.so: undefined reference to `mxIsCell'
../../../lib/Release/libMatlabIO.so: undefined reference to `mxCreateNumericArray@v7.0'
../../../lib/Release/libMatlabIO.so: undefined reference to `mxIsSingle@v7.0'
../../../lib/Release/libPartDetect.so: undefined reference to `mxGetCell'
../../../lib/Release/libMatlabIO.so: undefined reference to `matPutVariable@v7.0'
../../../lib/Release/libMatlabIO.so: undefined reference to `mxGetPr@v7.0'
../../../lib/Release/libPartDetect.so: undefined reference to `matPutVariable'
../../../lib/Release/libMatlabIO.so: undefined reference to `mxDestroyArray@v7.0'
../../../lib/Release/libMatlabIO.so: undefined reference to `matClose@v7.0'
../../../lib/Release/libMatlabIO.so: undefined reference to `matOpen@v7.0'
../../../lib/Release/libMatlabIO.so: undefined reference to `mxGetDimensions@v7.0'
../../../lib/Release/libMatlabIO.so: undefined reference to `matGetNextVariable@v7.0'
../../../lib/Release/libPartDetect.so: undefined reference to `mxSetCell'
../../../lib/Release/libMatlabIO.so: undefined reference to `mxGetNumberOfDimensions@v7.0'
../../../lib/Release/libPartDetect.so: undefined reference to `mxCreateNumericArray'
../../../lib/Release/libPartDetect.so: undefined reference to `mxCalcSingleSubscript'
together with this:
/usr/bin/ld: warning: libmx.so, needed by ../../../lib/Release/libMatlabIO.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libmat.so, needed by ../../../lib/Release/libMatlabIO.so, not found (try using -rpath or -rpath-link)

Solution:
I think the linker has a problem because these things are not in the library path. When I add include dir to it
; LD_LIBRARY_PATH=/home/frank/andriluca/matlab/extern/include it gives same errors
When I add matlab lib dir, as I think it should be since this dir has got the libmat.so in it, I get:
export LD_LIBRARY_PATH=/home/frank/andriluca/matlab/bin/glnx86/
/usr/bin/ld: error while loading shared libraries: libstdc++.so.5: cannot open shared object file: No such file or directory

So this is a new problem. I get this when I run any program too, gvim gives same error... This path must be wrong. Nope, I think it's a libstdc++ version
dependency problem. libmat and libmx from the matlab are depending on libstdc++.so.5 and I think I'm using 6.

New plan:
; sudo apt-get install libstdc++5
; export LD_LIBRARY_PATH=/home/frank/andriluca/matlab/bin/glnx86/
It seems to compile with warning about 5/6 version problem now.

Problem XI:
Now a system wide problem is this:
/home/frank/andriluca/matlab/bin/glnx86/libz.so.1: no version information
available (required by /usr/lib/libbfd-2.21.0-system.20110327.so)
So I think that the libs in the matlab thing are interfering with my system now. 

Solution:
I will remove LD_LIBRARY_PATH for now again and only add it when compiling this program

The program is now working. Go into the code_test directory to test (following the README):

Test the compiled code
- Issue the following commands in the code_test subdictory:
   ../run_partapp.sh --expopt ./expopt/exp_code_test.txt --part_detect --find_obj
   ../run_partapp.sh --expopt ./expopt/exp_code_test.txt --eval_segments
   This will estimate part positions on several example images and visualize the results.
   Compare the images in the ./log_dir/exp_code_test/part_marginals/seg_eval_images with the images in ./images_result

After comparing the images it seems they are very similar meaning the code works!

How to run the program on your own data

I am not sure how to do this yet. The commandline interface is not very user friendly. It is said that the command

   ../run_partapp.sh --expopt ./expopt/exp_code_test.txt --part_detect --find_obj

should be run in the code_test directory in order to create results in te directory log_dir. Though, the data that is in log_dir is being used by the application. Let’s start from the beginning:

; ../run_partapp.sh
argc: 2
argv[0]: ../bin/Release/partapp
reading command line parameters
command line options::
  --help                 help message
  --expopt arg           experiment parameters
  --train_class          train part detector
  --train_bootstrap      train part detector, adding hard negative samples
  --pidx arg             0-based index of the part
  --bootstrap_prep       create bootstrapping dataset (crops of objects with
                         some background)
  --bootstrap_detect     run previously trained classifer on bootstrapping
                         images
  --bootstrap_showrects  show top negatives on bootstrapping images
  --first arg            index of first image
  --numimgs arg          number of images to process
  --part_detect          run part detector on the test set
  --find_obj             find_obj
  --pc_learn             estimate prior on part configurations with maximum
                         likelihood
  --save_res             save object recognition results in al/idl formats
  --eval_segments        evaluate part localization according to Ferrari's
                         criteria
  --distribute           split processing into multiple chunks, keep track of
                         the chunks which must be processed
  --ncpu arg             number of chunks
  --batch_num arg        current chunk
  --vis_parts            visualize ground-truth part positions

From here, let’s start to try and run the part detection trainer on a set of images. The images we will be training on are the training images for the Ramanan people set, they are stored in the folder ramanan_people_train_h200. I expect to start the program to train it’s appearance model on these 200 images.

/home/frank/andriluka/partapp-r2/code_test2
; ../run_partapp.sh --train_class
'expopt' parameter missing

Let’s see what is needed in this experiment options file. When looking at the other expopt files in partapp-experiments-r2/expopt , these are the options that I found:

; cat abcparams4_trainall_rounds500_shape5_sparse_jb_radius24.txt
boosting_rounds: 500
desc_size: 24
desc_step: 12
window_desc_step_ratio: 1

num_train_pos: 212
num_train_jitter: 1138
num_train_neg: 1000
num_train_bootstrap: 350

feature_type: "SHAPEv5"

; cat exp_buffy_hog_detections.txt
train_dataset: "./ramanan_train_upperbody_h180.al"
validation_dataset: "./ramanan_train_upperbody_h180.al"
test_dataset: "./buffy_hog_detections.al"

part_conf: "./part_conf6_buffy_root1.txt"
abc_param: "./abcparams4_trainall_rounds500_shape5_sparse_jb_radius24.txt"
log_dir: "./log_dir"

min_object_scale: 1.0
max_object_scale: 1.0
num_scale_steps: 1

min_part_rotation: -180
max_part_rotation: 180
num_rotation_steps: 24

flip_orientation: false

scorehist_nbins: 200
scorehist_def_pw: 10
scorehist_def_ph: 10

object_height_width_ratio: 1.2

compute_part_marginals: true
num_pose_samples: 0
strip_border_detections: 0.3

 

I am now trying to use the first file to see if that is the data that is needed for training. Well…. no:

libprotobuf ERROR google/protobuf/text_format.cc:169] Error parsing text-format 
ExpParam: 1:16: Message type "ExpParam" has no field named "boosting_rounds".

So let’s try the other file? This is not the right approach… Let’s look in the code and use expopt/exp_code_test.txt as configuration file. Oh man I would kill for a better README..

<post not complete yet>