CV Chapter 6 Categorization 2

1. R-CNN 2. fast R-CNN and 3. faster R-CNN // R stands for regions

1. Extract ~2k region proposals from input image // Selective search

2. Compute CNN features out of warped region with pre-trained/fine-tuned network // AlexNet, VGGNet

3. Classify regions

1. Linear SVMs and 2. bounding box regressors (Bbox reg)

1. f_c(x_fc7) = w_c^T*x_fc7

2. With x_fc7 features from fully-connected layer 7 and c the object class

1. Predict 2D box due to wrong proposal region

2. Compute weight for new x*,y*,w* and h*

1. w^T_c,xx_pool5 = x*-x / w

2. w^T_c,yx_pool5 = y*-y / h

3. w^T_c,wx_pool5 = ln(w*/w)

4. w^T_c,hx_pool5 = ln(h*/h)

5. With x_pool5 features from pool5 layer

1. Fine tune network with softmax classifier (log loss)

2. Train post-hoc linear SVMs (hinge loss)

3. Train post-hoc bounding box regressors (squared loss)

4. Long and slow training (3 days and 47s per image)

5. High memory consumption (Easily 200GB)

1. Forward and 2. backward pass

1. Remove dependence on external region proposal algorithm

2. Get region proposal from same CNN

3. Use feature sharing

4. Single pass object detection becomes possible

Fast R-CNN + RPN

1. Total 4

2. Classification loss

3. Bounding box regression loss

1. All operations formulated as convolutions

2. Can process arbitrarily sized images

3. Forward with inference

4. Backward with learning

1. Perform pixel-wise prediction

2. Sliding-window classification producing heatmap of scores

3. Avoid low resolution with up-sampling and skip connections

1. Choose key-points with target disk (r) for skeleton joints

2. Each disk has ground-truth label of 1

1. Use triplet network with negatives, anchors and positives

2. Learn function grouping positives closer to anchors

3. Vector arithmetics possible due to linear regularities

1. Part-based models

2. Implicit shape models (ISM)

3. Deformable part-based model

1. Parts are 2D image fragments

2. Structure is configuration of parts

1. Bag of visual words with O(N)

2. Constellation with O(N^k)

3. Star shape with O(N²)

4. Tree with O(N²)

5. k-fan with O(N³)

6. Hierarchy

7. Sparse flexible model

1. Learn appearance codebook and star topology structural model

2. Features are considered independent given object center

3. Use visual vocabulary with displacement vectors to index votes

4. Robust to clutter, occlusion, noise and low contrast

1. Exact correspondence → probabilistic match

2. NN matching → soft matching

3. Feature location on object → part location distribution

4. Uniform votes → probabilistic vote weighting

5. Quantized hough array → continuous hough space

1. Choose interest points from image feature f

2. Compare with codebook entries with probabilistic vote weights

3. Locate object position and return back project hypothesis

1. Find pixel contributions with meta information from hypothesis

2. Perform segmentation

1. x_vote = x_img – x_occ*(s_img/s_occ)

2. y_vote = y_img – y_occ*(s_img/s_occ)

3. s_vote = (s_img/s_occ)

Each component has global template plus deformable parts // Bike

1. Use HOG sliding-window detector

2. Score is dot product of filter and vectors in window specified by p

3. Score of object hypothesis is sum of scores minus deformation costs s(p0,…,pn) = Sum_i Fi*phi(H,p_i) – Sum_i d_i*(dx_i²,dy_i²)

Bag-of-words model

1. Traditional recognition approach uses hand-designed feature extraction

2. Build better classifiers or more features?

3. New is learning features from pixel layer and forward to simple classifier

4. Inspiration by neuron cells

1. Multiple inputs x_1,… ,x_d

2. Multiple weights w_1,… ,w_d

3. Single output sigma(w*x+b) with sigma(t)=1/(1+exp(-t))

Input, hidden and output layer