siftGPU中文手册

SiftGPU Manual

Changchang Wu

University of North Carolina at Chapel Hill

Introduction

SiftGPU is a GPU implementation of David Lowe?s Scale Invariant Feature Transform.

SiftGPU是大卫·罗威的“S尺度不变特征变换”的一个GPU实现。

The following steps can use GPU to process pixels/features in a parallel way:

下面的步骤可以使用GPU来处理像素/特征用一种平行的方式：

1. Convert color to intensity, and up-sample or down-sample input images

将颜色信息转换成深度信息，对输入的影像进行升采样或者降采样

2. Build Gaussian image pyramids (Intensity, Gradient, DOG)

建立高斯图像金字塔（强度，梯度，DOG）

3. Keypoint detection (sub-pixel and sub-scale localization)

关键点检测（子像素和子尺度本地化）

4. Generate compact feature lists with GPU histogram reduction

生成直方图，利用GPU功能减少

5. Compute feature orientations and descriptors

计算特征的方位和描述子

By taking advantages of the large number of graphic processing units in modern graphic cards, this GPU implementation of SIFT can achieve a large speedup over CPU.

通过采取大量的图形处理单元在现代图形卡的优势，这款GPU实现SIFT的可以实现大加速了CPU。

Not all computation is faster on GPU, so this library also tries to find the best option for

each step. The latest version does intensity conversion, down-sampling, multi-orientation feature list rebuilding, and descriptor normalization on CPU. The latest keypoint list generation is also a GPU/CPU mixed implementation.

并不是所有的计算是更快GPU上，所以这个库还试图找到用于每个步骤的最好的选择。最新版本确实强度转换，下采样，多方位的功能列表重建，并在CPU做描述子的归一化。最新的关键点列表生成也是GPU / CPU混合实施。

Running SiftGPU requires a high-end graphic card that 1) has a large graphic memory to keep the allocated intermediate textures for efficient processing of new images. 2) Supports dynamic branching. The loops in orientation computation and descriptor

generation are decided by the scale of the features, and they cannot be unrolled.

SiftGPU now runs on GLSL by default, which works for both ATI and nVidia. You can

optionally use CUDA for nVidia graphic cards.

运行SiftGPU需要高端的显卡：1）有一个大的图形内存，以保持所分配的中间纹理的新的图像有效处理。2）支持动态分支。在取向计算和描述符生成回路通过的特征的规模决定，并且它们不能被展开。SiftGPU现在在默认情况下，同时适用于ATI和nVidia运行在GLSL。您可以选择使用CUDA的nVidia的显卡。

Interface: class SiftGPU

Class SiftGPU is provided as the interface of this library. The following examples will show how to use this class to run SIFT in a different ways.

类SiftGPU是作为该库的接口。下面的例子将展示如何使用这个类在一个不同的方式运行SIFT。Initialization, Normal initialization for an application

初始化，正常初始化应用程序

//create a SiftGPU instance

//创建一个SiftGPU实例

SiftGPU sift;

//processing parameters first

//首先处理参数

char * argv[] ={ "-fo", "-1", “-v”, “1”};

//-fo -1, //-v 1, starting from -1 octave 从-1八度开始

only print out # feature and overall time 只打印出来特征点和总时间

sift.ParseParam(4, argv);

//create an OpenGL context for computation

创建一个OpenGL文件进行计算

int support = sift.CreateContextGL();

//call VerfifyContexGL instead if using your own GL context 如果不是用你自己的GL文件，那么就调用VerfifyContexGL

//int support = sift.VerifyContextGL();

if(support != SiftGPU::SIFTGPU_FULL_SUPPORTED) return;

Example #1, run sift on a set of images and get results:

例子1：在一组影像中运行sift，并得到结果

//process an image, and save ASCII format SIFT files

处理一张图像，并保存ASCII格式的SIFT文件

if(sift.RunSIFT("1.jpg"))

sift.SaveSIFT("1.sift");

//you can get the feature vector and store it yourself

你可以得到特征向量，并将它保存起来

sift.RunSIFT("2.jpg");

int num = sift.GetFeatureNum();//get feature count获取特征数量

//allocate memory for readback

分配内存，以方便回读

vector descriptors(128*num);

vector keys(num);

//read back keypoints and normalized descritpros

回读特征点，并且归一化描述子

//specify NULL if you don’t need keypionts or descriptors 如果你不需要特征点或者描述子，那么将制定为NULL

sift.GetFeatureVector(&keys[0], &descriptors[0]);

Example #2, run SiftGPU with your own image data

例子2, 运行SiftGPU用自己的图像数据

// This is very convenient for camera application

这对于摄像机是非常方便的

int width = …, height =…

unsigned char *data = … // your (intensity) image data你的影像数据sift.RunSIFT (width, height, data, GL_RGBA, GL_UNSIGNED_BYTE);

//Using GL_LUMINANCE data saves transfer time

//使用GL_LUMINANCE数据保存传输时间

Example #3, specify a set of image inputs using SetImageList

例子3，使用SetImageList指定一组图像输入

char * files[4] = { “1.jpg”, “2.jpg”, “3.jpg”, “4.jpg”};

sift.SetImageList(4, files);

//Now you can process an image with its index

现在你可以根据某张影像的索引，来运行它

sift.RunSIFT(1);

sift.RunSIFT(0);

Example #4, control storage allocation

例子4，控制内存分配

//Option1, use“-p”, “1024x1024” to initialize the texture

//选项1，使用“-p”，“1024×1024”初始化纹理

//storage for size 1024x1024, so that processing smaller

//存储大小1024×1024，以便处理更小的

//images does not require texture re-allocation

//图像不需要纹理重新分配

//char * argv[] ={ "-m", "-s", “-p”, “1024x1024”};

//sift.ParseParam(4, argv);

//Option2, manually allocate the storage

//选项2，手动分配存储

sift.AllocatePyramid(1024, 1024);

// processing images with different sizes.

根据不用的尺寸来处理影像

sift.RunSIFT(“1024x768.jpg”);

sift.RunSIFT(“768x1024.jpg”);

sift.RunSIFT(“800x600.jpg”);

Example #5, runtime library loading

例子5，运行库加载

//new exported function CreateNewSiftGPU

新的导出函数CreateNewSiftGPU

SiftGPU* (*pCreateNewSiftGPU)(int) = NULL;

//Load siftgpu dll… use dlopen in linux/mac

//加载siftgpu DLL ...使用的dlopen在Linux / MAC

HMODULE hsiftgpu = LoadLibrary("siftgpu.dll");

//get function address

获取函数地址

pCreateNewSiftGPU = (SiftGPU* (*) (int))

GetProcAddress(hsiftgpu, "CreateNewSiftGPU");

//create a new siftgpu instance

//exported functions are all virtual

导出的函数都是虚函数

SiftGPU * psift = pCreateNewSiftGPU(1);

Example #6, Compute descriptor for user-specified keypoints

例子6，为用户指定的关键点，计算描述子

vector keys;

//load your sift keypoints using your own function 运用自己的函数来加载sift特征点

LoadMyKeyPoints(…);

//Specify the keypoints for next image to siftgpu

指定特征点为下一张影像的特征点

sift.SetKeypointList(keys.size(), &keys[0]);

sift.RunSIFT(new_image_path);// RunSIFT on your image data //****If it is to re-run SIFT with different keypoints***

如果是重新运行SIFT不同的关键点，用。。。。跳过过滤

//Use sift.RunSIFT(keys.size(), &keys[0]) to skip filtering

//Get the feature descriptor

float descriptor* = new [128 * keys.size()];

//We only need to read back the descriptors

我们只需要回读描述子

sift.GetFeatureVector(NULL, descriptor);

Example #7, Compute (guided) putative sift matches.

例子7，计算（引导）嘉定的sift匹配

//specify the naximum number of features to match

指定特征的数量的最大值，来进行匹配

SiftMatchGPU matcher(4096);

//You can call SetMaxSift anytime to change this limit

你也可以运用SetMaxSift，随时来更改这个限制

//You can call SetLanguage to select shader language

你可以在初始化之前用SetLanguage，来选择着色器语言在GLSL/CUDA之间

//between GLSL/CUDA before initialization

//Verify current OpenGL Context and do initialization

//验证当前的OpenGL文件并做初始化

if(matcher.VerifyContextGL() == 0) return;

//Set two sets of descriptor data to the matcher

设置两组描述子数据，以到匹配

matcher.SetDescriptors(0, num1, des1);

matcher.SetDescriptors(1, num2, des2);

//Match and read back result to input buffer

匹配和回读结果，输入缓冲区

int match_buf[4096][2];

int nmatch = matcher.GetSiftMatch(4096, match_buf);

// You can also use homography and/or fundamental matrix to // guide the putative matching

你也可以利用单应矩阵或者基础矩阵，来引导假定的匹配

// Check function SiftMatchGPU::GetGuidedMatch

// For more details of the above functions, check

// SimpleSIFT.cpp and SiftGPU.h in the code package.

Example #8, Choosing the GPU for computation (under Multi-GPU system)

例子8，选择GPU进行计算（在多GPU系统）

//Suppose (1024,0) is in the screen of the second GPU //假设（1024,0）是在第二GPU的屏幕

//For CUDA, use “-cuda”, “device_index”

//For Windows and X11, use “-display”, “display_name”

char * argv[] ={"-fo", "-1", “-display”, “\\\\.\DISPLAY4”};

siftgpu->ParseParam.(4, argv);

if(!siftgpu->VerifyContextGL) return;

//GPU selection can’t be changed after VerifyContextGL

Example #9, Using a local multi-process mode NEW

例子9，利用一个本地的多线性模式NEW

//1st parameter, 7777 for the socket port used by server

//第一个参数7777，用于服务器的插座段

//2nd parameter, NULL for local process mode

//第2个参数，NULL用于本地处理模式

SiftGPU* siftgpu = new ServerSiftGPU(7777, NULL);

//You can create multiple ServerSiftGPU instances, and

//let them use different GPUs.

你可以创建多个ServerSiftGPU实例，并让他们使用不同的GPU

//Everything else is the same.

其他的所有都一样

siftgpu->ParseParam(…);//choose GPU if more than one

if(!siftgpu->VerifyContextGL()) return;

//Call RunSIFT functions. Most functions are supported

//调用RunSIFT功能。大部分功能都支持

siftgpu->RunSIFT("1.jpg");

//when you call delete, the server will be shut down.

//当你调用删除时，服务器将关闭

delete siftgpu;

Example #10, Using a remote SiftGPU on different computer NEW

例子10，在不同的机器上使用一个远程的SiftGPU

//Suppose, you have a computer at https://www.sodocs.net/doc/d42096334.html,

//you first start a siftgpu server by run

// bin/server_siftgpu –server 7777 [siftgpu param]

//From a different computer you can do as follows

SiftGPU* siftgpu = new ServerSiftGPU(7777, “https://www.sodocs.net/doc/d42096334.html,”);

siftgpu->ParseParam(…)

//if GPU selection is already done on the server.

//New GPU selection won’t work

if(!siftgpu->VerifyContextGL()) return;

delete siftgpu;

//after delete, server keeps running and accept new clients

OpenGL Context

SiftGPU uses OpenGL (Not for the new multi-process mode and remote mode), and there

has to be an OpenGL context to run the program. There are several ways to initialize the OpenGL context:

SiftGPU使用OpenGL（不是新多进程模式和远程模式），并且必须有一个OpenGL上下文来

运行程序。有几种方法来初始化OpenGL上下文：

1. Use function SiftGPU::CreateContextGL. It uses Win32/XLib (or GLUT depending

on your compilation setting) to create an invisible window and use that GL context

to run the shaders. (The example SimpleSIFT is doing this way).

1、使用功能SiftGPU :: CreateContextGL。它使用的Win32 /XLib（或GLUT取决于你的编译环境）

来创建一个不可见的窗口，并使用GL文件来运行着色器。（这个例子SimpleSIFT是这样做

的方式）。

2. Use GLUT yourself (see example project TestWinGlut).

3. Use raw OpenGL functions (see example project TestWin). You have to make

sure you have an active context before calling SiftGPU functions (for example: call

WglMakeCurrent to set the context in windows).

使用原OpenGL函数（见示例项目TestWin）。你必须确保你有一个活跃的文件在调用SiftGPU

功能前（例如：加载WglMakeCurrent来设置窗口中的上下文中）。

Multiple Implementations (GLSL/CUDA)

The code package includes 3 different implementations of SiftGPU. GLSL Unpacked/Packed and CUDA. They can be selected by using combination of “-glsl”, “- unpack”, “-pack” and “-cuda”. “-glsl -pack” is now default.

该代码软件包包括3个不同的SiftGPU的实现。GLSL未封装/封装和CUDA。它们可以通过使

用“-glsl”，“-unpack”，“-pack”和“-cuda”，“-glsl -pack”的组合来实现，现在是默认的。

The processing speed decreases when the image size increases. On NVIDIA 8800 GTX,

the GLSL packed version is faster than CUDA for large images, but CUDA is faster for

small images. This order could be different on different GPUs, and you can just try them

on your computer to select the best one for different image sizes.

当图象尺寸的增加，处理速度下降。在NVIDIA 8800 GTX下，对于大影像来说GLSL包装版本比CUDA快，但对于较小的影像CUDA更快。这个命令可能在不同的GPUs下不同，你可以尝试一下您的计算机上，

根据不同大小的图像选择最适合的。

SiftMatchGPU also has implementations for GLSL and CUDA, and they can be selected

by calling function SiftMatchGPU::SetLanguage. GLSL matching is slightly slower than CUDA for exhaustive putative matching. GLSL matching is faster for guided putative matching.

SiftMatchGPU还具有实现对GLSL和CUDA，并且它们可以通过调用函数SiftMatchGPU :: SetLanguage 来选择。对于详尽的假定匹配GLSL匹配比CUDA稍慢了，对于指定的假设匹配GLSL匹配更快些

SiftGPU for Multiple-GPU

Device Selection: You can select a particular GPU for SiftGPU computation. Different method need to be used in different systems and different implementations.

设备选择：您可以选择SiftGPU计算特定的GPU。在不同的系统和不同的实现中使用不同的方

法的需要。

When using CUDA-based SiftGPU, you need to set parameter “-cuda device_index” when

you want to use a particular device. For SiftMatchGPU, you need call SiftMatchGPU:: SetLanguage(SIFTMATCH_CUDA + device_index)

当使用基于CUDA的SiftGPU，你需要设置参数“-cuda device_index”当你想使用一个特定的设备。

对于SiftMatchGPU，你需要调用SiftMatchGPU :: SetLanguage（SIFTMATCH_CUDA + device_index）

When using OpenGL-based implementation under Win32, you need to specify a device

that is obtained by EnumDisplayDevices ( for example, “-display”, “\\\\.\\DISPLAY4”). This

is now tested under WIN7 on a machine that has both ATI and nVidia cards.

当使用Win32下基于OpenGL的实现，需要指定由EnumDisplayDevices获得的设备

（例如，“-display”，“\\\\。\\ DISPLAY4”）。这是现在WIN7下的机器，同时具有ATI和NVIDIA显

卡上测试。

When using OpenGL-based implementation under X-Window, you need use parameter

“-displa y hostname:number.screen_number” to select a display to let SiftGPU use the corresponding GPU for computation.

当使用在X-Window的基于OpenGL的实现，需要使用参数“-display主机名：number.screen_number”选择显示让SiftGPU使用相应的GPU进行计算。

Multiple SiftGPU instances

Note that CUDA version can be multi-threaded if each thread is setup to use

different device. See MultiThreadSIFT for an demo of this.

需要注意的是CUDA的版本可以是多线程的，如果每个线程都设置为使用不同的设备。

见MultiThreadSIFT对于这样的一个演示。

It is hard for the OpenGL-based one to use the multiple GPUs in the same process. However, you can run multiple GPU programs in different process to utilize

different GPUs. The new version of SiftGPU is able to simply work as a client and controls multiple worker processes. It is able to automatically create worker processes on local computer and connect to some existing server on local/remote

computer.

这是很难的基于OpenGL之一来使用多个GPU中相同的过程。但是，可以在不同的进程中运行多

个GPU方案，利用不同的GPU。SiftGPU新版本能够简单地作为一个客户端，并控制多个工作进程。它能够自动创建本地计算机上的工作进程，并连接到本地/远程计算机的一些现有的服务器。

SiftGPU includes an implementation of SiftGPU server [server_siftgpu in

server.cpp] and SiftGPU client [class ServerSiftGPU], with which you can easily

run multiple SiftGPU instances on different GPUs on your local computer, or use a remote computer to run all the computation. What?s most important is that it

doesn?t change any of the programming interfaces, and all the wrapping is done

internally.

SiftGPU包括实施SiftGPU服务器[server_siftgpu在server.cpp中]和SiftGPU客户端[类ServerSiftGPU] 的，使用它可以很容易地在本地计算机上运行在不同的GPU多SiftGPU实例，或者使用远程计算机

上运行所有计算。最重要的是它没有改变任何的编程接口，所有的包装是内部完成。

You can look at server.cpp for examples. It is not only the implementation of the

several but also includes some client-server example. The two command line

options “-test” and “-test2”gives you the two examples.

你可以看一下server.cpp中的例子。它不仅是several的执行，而且还包括一些客户端- 服务器的例子。这两个命令行选项“-test”和“-test2”给你两个例子。

Memory Management

SiftGPU needs to allocate OpenGL textures (or CUDA linear memory/texture) for storing intermediate results. This allocation is a time-consuming step, and it would be efficient if

memory re-allocation is infrequent and the storage can be re-used to process lots of

images. The best performance can be obtained when you pre-resize all images to a same

size, and process them with one SiftGPU instance.

SiftGPU需要分配OpenGL纹理（或CUDA线性存储器/纹理）存储中间结果。这种分配是一个耗时

的步骤，这将是有效的，如果存储器重新分配是罕见的，且存储可以被重新用于处理大量的图像。最佳的性能可以得到，当你预先调整所有的影像到相同的大小，并有一个SiftGPU实例处理。

When starting up, you can pre-allocate the memories to fit some specified size or SiftGPU

will automatically fit the first image. You can also manually re-allocate the active pyramid

at anytime by calling SiftGPU::AllocatePyramid(int w idth, int height).

启动时，你可以预先分配内存，以适应某些特定的大小或SiftGPU会自动适应第一张图像。

您也可以手动重新分配活跃的金字塔，通过调用SiftGPU :: AllocatePyramid（int width，int height）。

While processing an image that has a different size, the storage by default will

automatically resize to fit the largest width and the largest height so far. But you can pre-

allocate it to the largest size you know so that there won?t be any re-allocation. SiftGPU

reuses existing storage to process any smaller images that can fit in (See example 4).

在处理具有不同尺寸的图像时，存储在默认情况下会自动调整迄今以适应最大宽度和最大高度。

但是你可以预先分配最大尺度的内存，所以这将会没有任何重新分配。SiftGPU利用现有

的存储过程，可以适应任何较小的图像（见例4）。

Optionally, you can select a tight mode by calling function SiftGPU::SetTightPyrmid(int

tight = 1). The storage will then resize to any new image size. It does save memory for

smaller images, but there will be a re-allocation each time when the image size changes.

或者，您可以通过调用SiftGPU::SetTightPyrmid(inttight = 1)选择一tight模式。存储将会调整到

任何新的图像大小。对于较小的影像它可以节省内存，但是它会有一个重新分配，当影像大小改变时。NOTE: When you run TestWinGlut with the first input image, it will print out the total

number of megabytes of textures it takes (not including the copy of the original 4-channel image). Please do compare that number with your total number of GPU memory.

注意：当您运行TestWinGlut与第一输入图像，它会打印出需要的纹理的兆字节总数（不包括原来的4

通道图像的复印件）。请不要与GPU内存的总次数比较这个数字。

Parameter System (used by SiftGPU::ParseParam)

? the parameter can be changed after initialization in all implementations

? the parameter can be changed after initialization in CUDA implementation

SiftGPU Viewers

There are 2 GUI viewers for SiftGPU

TestWinGlut is a GLUT-based viewer

TestWin.exe directly uses Win32 API to control OpenGL Contexts

There are 7 view modes in the viewers:

0, original image and feature (drawn as blue points or rectangles):

1, Gaussian pyramid

2, octaves (View different octaves one by one)

3, levels (View different levels one by one)

4, the pyramid of difference of Gaussian

5, the pyramid of image gradient

6, detected keypoints in levels. Red points are the local maxima, and green points are local minima. You can zoom to see the details in levels

Viewer keys

You can loop through these view modes by pressing the following keys:

Enter, Backspace, Space . (>) , (<)

x, Escape next view

previous view

next sub-view/level/octave (in view mode 0, 2, 3) previous sub-view/ level/octave (in view mode 0, 2, 3) exit

Some other controls are as follows:

Mouse r

o

+, = -,

l

c

q click, hold and move to pan the view

Go to the next image if there is, and re-compute SIFT

reset coordinate

zoom in

zoom out

start/stop l oopy processing of a set of images

Randomize the c olors for sift feature box display in view mode 0-2 Change verbose level 2-1-0-2-1-0…

Demos

1. There are three demo batch files in the …demos? folder.

Demo1.bat is a basic example of SiftGPU. Try step through all the views to see

the intermediate results of SIFT using the controls explained in the last section.

Demo2.bat shows the processing of a file that contains a list of image filenames.

The images in this demo are all of size 640x480. After the viewer shows up, press …l? to start/stop process the input images one by one repeatedly. Other keys also

work to change the view modes during the loop.

Demo3.bat shows the processing of a list of images of varying sizes.

Evaluation-box.bat computes the sift features for comparing with Lowe?s result.

2. SimpleSIFT project in the workspace/solution shows how to use SiftGPU without GUI.

It also shows how to read back SIFT results from SiftGPU. There is also an optional macro which enables runtime loading of SiftGPU library.

3. Speed project shows how to evaluate the speed of SiftGPU

4. ServerSiftGPU shows how to use SiftGPU as a computation server. The file

/src/ServerSiftGPU/server.cpp gives the implementation of the server. With argument “-test” and “-test2” it can run as a client, which demos the usage of ServerSiftGPU.

5. MultiThreadSIFT shows how to multi-thread SiftGPU and use multiple GPU devices.