Yuhang He's Blog

Some birds are not meant to be caged, their feathers are just too bright.

compute top N error with Matlab in Caffe

Top-N error rate is usually applied to measure how successfully a set of images with the same class name is classified by a pre-trained Caffe model. First, we have to figure out the meaning of Top-N error rate: Top-N error rate indicates the fraction of image number that does not appear in the top N predicted results by the pre-trained model. According to this definition, we have analysis all the predicted results for each image, when it comes to maltab, several functions are helpful: find(), sort() and length().

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
function [ top_N_error ] = compute_topN_error( img_list, category_id )
  fid = fopen( img_list, 'r' );
  %get the file lines number
  line_num = 0;
  while ~feof( fid )
    line_num = line_num + sum( fread(fid, 10000,'*char') == char(10) );
  end
  fclose( fid );
  top_N_error = 0;
  fid = fopen( img_list, 'r' );
  class_rst_mat = zeros( line_num, 6 );
  fid_record = fopen('/mnt/lvm/category_id.txt','w');
  for i = 1:line_num
    img_dir = fgetl(fid);
    img = imread( img_dir );
    class_rst_mat(i,:) = classification_demo( img, 0 );
  end
  fclose( fid_record );
  [ class_rst_mat1, class_rst_mat2 ] = sort( class_rst_mat, 2 );
  top_N_error = zeros( 1, 5 );
  for i = 1:5
    num_classified = length( find( class_rst_mat2(:,(7-i):6) == category_id ) );
    top_N_error(1,i) = 1 - num_classified/line_num;
  end
  fclose( fid );
end

Note that, I first compute the image number from the image_list file, which builds on the fact that the ASCII code of '\n' is 10. Besides, I use sort( mat, 2) to guarantee the sorting process is row-majored. The returned sorted matrix is in largest-in-right format. The combination of length() and find() ensures to return the number of top-N images that successfully classified.

The function classification_demo() is a pre-implemented function in Caffe. It accepts an image and a flag indicating whether to compute on CPU or GPU, and returns a row vector indicating the probability of the image being classified to each class. Of course, it also allows to process multiple images at the same time by concatenating each image in row-majored forms.

There are something needed to be careful. Matlab stores image in RGB order, column-major. However, caffe requires BGR, row-majored data input. That is, we have to pre-process the input image with the following code:

1
2
3
img_cnt = img(:, :, [3, 2, 1]);
img_cnt = permute( img_cnt, [2, 1, 3] );
img_cnt = single( img_cnt );