camera interface
Generic camera interface for image acquisition.
Multiple image (multiple out) output ports can be published, organized into
individual banks. Each bank contains a sequence of coherent images,
e.g. time synchronized.
The internal architecture implements a graph of named source and sink
slots, which can be published individually into images of a bank.
Hardware-dependent functions can stream raw data into slots, which
can then filtered or otherwise processed by generic services
into other slots and eventually published with publish (activity).
Note that the term slot in this documentation always refers to an
internal image of the processing graph, while the term bank refers
to a published image in a bank of the port image (multiple out).
Ports
image (multiple out)
Data structure
|
Published image banks.
Images in a bank are considered as a coherent sequence of individual
frames, hence sharing a common ts timestamp information.
Each image in the array contains its own metadata information
info and a pixmap. The seq member represents a sequence
number and different images read from the same bank index with the
same seq number should be considered identical. This is especially
useful to check if the calibration data has changed.
info.roi indicates that the image data was extracted from a
bigger image. In particular, the intrinsics refer to the original
image dimensions.
info.distortion represents the image distortion information and
the associated correction model.
info.intrinsics is the intrinsic matrix mapping from 2D
coordinates to 3D information, according to the following model where
u,v are image pixel coordinates, w a free parameter and X,
Y, Z 3D coordinates in the camera frame:
| w.u | fx 0 cx | | X | w.v = | 0 fy cy |.| Y | w | 0 0 1 | | Z
info.extrinsics is a 3D transformation converting 3D coordinates
from the camera frame to another frame common to all images in a
bank:
| X' | X | Y' = R.| Y + T | Z' | Z
pixmap.data contain the actual pixel values, arranged as a
row-major 2D array of height×width pixels. The width in bytes
of each pixel row is indicated by stride.
Each pixel has the format pixmap.pixfmt, defining the number of
channels: MONO is 1 integer grayscale channel, RGB or BGR are
3-color integer channels, possibly with an additional transparency
channel A. Z and XYZ are 1 and 3 floating point depth channels
repectively. The number of bits encoding each channel value is
suffixed. JPEG indicates a compressed RGB color image.
Services
publish (activity)
Inputs
|
Throws
|
Context
|
Start publishing a named slot to an output port.
This exports the internal image present in the named slot into the
output port bank, at the position index within the sequence of
images of the port (see image (multiple out) structure).
slot must be the name of an output slot of a hardware streaming
function or a software processing function. bank name can be chosen
freely and is reflected in the output port name. index can also be
chosen freely depending on the desired configuration of the output
port.
If multiple images have to be published in a time-synchronized
manner, they should be placed in the same bank, at different index
position. For instance, if to slots left and right are streamed
by a stereoscopic hardware, they can be placed in the same port named
stereo by using two publish requests with slot left, bank
stereo, index 0 and then slot right, bank stereo, index 1.
If multiple images have to be published but do not require specific
time syncronization, they can be published to different ports. This
can make the client access more efficient, as each client can read
only the images in which it is interested in. For instance, to
publish the above example images left and right to port A and
B, use two publish requests with slot left, bank A, index 0
and then slot right, bank B, index 0.
unpublish (activity)
Inputs
|
Throws
|
Context
|
Stop publishing a port.
See publish (activity).
jpeg (function)
Inputs
|
Throws
|
Compress a slot using JPEG compression.
This converts the input slot to the output slot output using
JPEG compression, with the specified quality.
record_mpeg (function)
Inputs
|
Throws
|
Record a slot using MPEG compression in an AVI container.
The input slot is first resized to the desired width and
height and compressed into the output file. The playback video
frame rate is fixed and equal to fps, no matter the input slot
rate. The output file name must end with .avi.
Additionnaly, an output slot named after the output file name, with
any leading slash (/) stripped, contains the last compressed image,
appropriately scaled to width and height.
cancel (function)
Inputs
|
Throws
|
Cancel all jobs publishing to slot.
The effect of this function is that slot will no longer be updated
and the associated processing will be cancelled. This is useful to
reconfigure the processing graph with e.g. different parameters or
different inputs.
status (function)
Inputs
|
Throws
|
Raise any processing errors associated with slot.
If, for some reason, a slot fails to be updated, the error will be available by checking the result of this function. Processing errors typically include wrong pixel format for a filter or wrong image dimensions.
Tasks
worker
Context
|
Worker task processing and publishing frames.