=====================================================================================================
PREPROCESS STAGE
-----------------------------------------------------------------------------------------------------
Input is specified in Matlab file relighting.m (type "help relighting" in Matlab interpreter).
Preprocess stage takes about 5 hours for the example of the David in an Intel dual processor
Xeon with 4GB of memory.
The most costly operation (time and memory space) is the first SVD for each color component. SVD
could be implemented in C to avoid matrix duplication as Matlab does when performing an SVD.
Information from the function is reproduced here:
function relighting(myPath, myLightRes, myBlocksW, myBlocksH, myBlockSize, myRank, myRank2)
% Process first and second stages of PCA for the input set of images and
% given parameters.
%
% ATTENTION : This function requires a lot of memory, take care with the
% parameters that you choose.
%
% INPUT
% Use as: relighting(myPath, myLightRes, myBlocksW, myBlocksH,
% myBlockSize, myRank, myRanks2)
%
% Where:
% myPath : Path to the set of input images.
% Example: 'c:/senrique/relighting/nicoleb6x256/final/'
% Path must include the final slash to work properly.
% Input images should be jpg files with the name:
% 'file(f)(uu)(vv).jpg' -without parenthesis-
% .
% Where:
% f : Face, 0: +X, 1: -X, 2: +Y, 3: -Y, 4: +Z, 5: -Z.
% +X points to the right, +Y points to the sky, +Z points
% to you.
% uu, vv : 00..myLightRes-1
% if f=0 then uu is from +Y to -Y,
% vv is from +Z to -Z
% f=1 then uu is from +Y to -Y,
% vv is from -Z to +Z
% f=2 then uu is from -Z to +Z,
% vv is from -X to +X
% f=3 then uu is from +Z to -Z,
% vv is from -X to +X
% f=4 then uu is from +Y to -Y,
% vv is from -X to +X
% f=5 then uu is from -Y to +Y,
% vv is from -X to +X
% Example filename: file10314.jpg
% myLightRes : Each face was divided in a grid of myLightRes x
% myLightRes to generate the corresponding picture with a
% point light source. There are 6 x myLightRes x myLightRes
% image files.
% Example: 16
% myBlocksW : Each image is composed by myBlocksE of width.
% Example: 32
% myBlocksH : Each image is composed by myBlocksH of height.
% Example: 32
% myBlockSize : Each block is composed of myBlockSize x myBlockSize
% pixels.
% Example: 16
% (myBlocksW * myBlockSize) should be the image width.
% (myBlocksH * myBlockSize) should be the image height.
% In this example, width = height = 32 x 16 = 512
% myRank : Rank to use in the first PCA computation.
% Example: 10
% myRank2 : Rank to use in the second PCA computation.
% Example: 200
%
% OUTPUT
% The following files will be generated in the working directory:
%
% - Files for input in the Relighting Framework (Real-Time):
% Er.txt
% Eg.txt
% Eb.txt
% Ur.txt
% Ug.txt
% Ub.txt
% Vr.txt
% Vg.txt
% Vb.txt
% --> see LSD paper for details about the meaning of these files.
%
% - Additional files:
% status.mat : Some variables used for the calculation and for debugging
% purposes, like initial and final time, elapsed time, etc.
% Ir.mat, Ig.mat, Ib.mat : All the original images packed in a big
% matrix.
% For each .txt file generated, the corresponding .mat file.
%
% ---
% Sebastian Enrique - senrique@cs.columbia.edu
% Columbia University - January 2004
=====================================================================================================
=====================================================================================================
REALTIME STAGE
-----------------------------------------------------------------------------------------------------
Input for realtime stage are the files:
% Er.txt
% Eg.txt
% Eb.txt
% Ur.txt
% Ug.txt
% Ub.txt
% Vr.txt
% Vg.txt
% Vb.txt
Each file contains a matrix of [m x n] (where m and n change in each matrix) of float values in
ASCII format. Each row is a matrix row from 0 to m (separated by carriage return and line feed)
and in each row you can find the n values separated by space.
There are three files -one for each color component- for matrices E, U and V.
Recall that using SVD: Ij ~= Ej Sj t(Cj)
Where:
Ij is a [p x n] matrix representing a collection of image blocks, the I column of Ij is
formed by p pixels of the jth block. (pixels from the block are specified from left to
right and then top to bottom)
Ej is a [p x b] column-orthogonal matrix, called block bases.
Sj is a [b x b] diagonal matrix.
Cj is a [n x b] column-orthogonal matrix.
t(X) is the transpose of matrix X.
p is the quantity of pixels per block.
b is the rank used for the svd.
n is the quantity of images.
E contains (myBlocksW * myBlocksH) submatrices (one for each block) stacked one below the other.
The order of the blocks is from left to right and then from top to bottom. Each submatrix Ej is
a [p x b] matrix, so E is a [p*myBlocksW*myBlocksH x b] matrix.
Lets say that: Ij ~= Ej Sj t(Cj) = Ej Lj
Where:
Lj is called block lighting coefficients, a [b x n] matrix. We stacked all Lj matrices
in a big matrix L in the same way than E. So, L is a [b*myBlocksW*myBlocksH x n] matrix.
Using SVD we on L we can get: L ~= U S t(T) = U V
Where:
U is a [b*myBlocksW*myBlocksH x q] matrix, called lighting coefficient bases.
V is a [q x n] matrx, called compressed coefficient matrix.
q is the rank used for the this second svd.
=====================================================================================================