libcamera: libcamera::ipa::ipu3::IPAIPU3 Class Reference

libcamera v0.5.1 Supporting cameras in Linux since 2019

Loading...
Searching...
No Matches
libcamera::ipa::ipu3::IPAIPU3 Class Reference

The IPU3 IPA implementation. More...

Inheritance diagram for libcamera::ipa::ipu3::IPAIPU3:
[legend]
Collaboration diagram for libcamera::ipa::ipu3::IPAIPU3:
[legend]

## Public Member Functions
int | init (const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) override
| Initialize the IPA module and its controls.
int | start () override
| Perform any processing required before the first frame.
void | stop () override
| Ensure that all processing has completed.
int | configure (const IPAConfigInfo &configInfo, ControlInfoMap *ipaControls) override
| Configure the IPU3 IPA.
void | mapBuffers (const std::vector< IPABuffer > &buffers) override
| Map the parameters and stats buffers allocated in the pipeline handler.
void | unmapBuffers (const std::vector< unsigned int > &ids) override
| Unmap the parameters and stats buffers.
void | queueRequest (const uint32_t frame, const ControlList &controls) override
| Queue a request and process the control list from the application.
void | computeParams (const uint32_t frame, const uint32_t bufferId) override
| Fill and return a buffer with ISP processing parameters for a frame.
void | processStats (const uint32_t frame, const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) override
| Process the statistics generated by the ImgU.
Public Member Functions inherited from libcamera::ipa::Module< IPAContext, IPAFrameContext, IPAConfigInfo, ipu3_uapi_params, ipu3_uapi_stats_3a >
const std::list< std::unique_ptr< Algorithm< Module > > > & | algorithms () const
| Retrieve the list of instantiated algorithms.
int | createAlgorithms (Context &context, const YamlObject &algorithms)
| Create algorithms from YAML configuration data.

## Protected Member Functions
std::string | logPrefix () const override
| Retrieve a string to be prefixed to the log message.
Protected Member Functions inherited from libcamera::Loggable
LogMessage | _log (const LogCategory *category, LogSeverity severity, const char *fileName=__builtin_FILE(), unsigned int line=__builtin_LINE()) const
| Create a temporary LogMessage object to log a message.

## Additional Inherited Members
Public Types inherited from libcamera::ipa::Module< IPAContext, IPAFrameContext, IPAConfigInfo, ipu3_uapi_params, ipu3_uapi_stats_3a >
using | Context
| The type of the shared IPA context.
using | FrameContext
| The type of the frame context.
using | Config
| The type of the IPA configuration data.
using | Params
| The type of the ISP specific parameters.
using | Stats
| The type of the IPA statistics and ISP results.
Static Public Member Functions inherited from libcamera::ipa::Module< IPAContext, IPAFrameContext, IPAConfigInfo, ipu3_uapi_params, ipu3_uapi_stats_3a >
static void | registerAlgorithm (AlgorithmFactoryBase< Module > *factory)
| Add an algorithm factory class to the list of available algorithms.

## Detailed Description

The IPU3 IPA implementation.

The IPU3 Pipeline defines an IPU3-specific interface for communication between the PipelineHandler and the IPA module.

We extend the IPAIPU3Interface to implement our algorithms and handle calls from the IPU3 PipelineHandler to satisfy requests from the application.

At initialisation time, a CameraSensorHelper is instantiated to support camera-specific calculations, while the default controls are computed, and the algorithms are instantiated from the tuning data file.

The IPU3 ImgU operates with a grid layout to divide the overall frame into rectangular cells of pixels. When the IPA is configured, we determine the best grid for the statistics based on the pipeline handler Bayer Down Scaler output size.

Two main events are then handled to operate the IPU3 ImgU by populating its parameter buffer, and adapting the settings of the sensor attached to the IPU3 CIO2 through sensor-specific V4L2 controls.

In computeParams(), we populate the ImgU parameter buffer with settings to configure the device in preparation for handling the frame queued in the Request.

When the frame has completed processing, the ImgU will generate a statistics buffer which is given to the IPA with processStats(). In this we run the algorithms to parse the statistics and cache any results for the next computeParams() call.

The individual algorithms are split into modular components that are called iteratively to allow them to process statistics from the ImgU in the order defined in the tuning data file.

The current implementation supports five core algorithms:

- Auto focus (AF)
- Automatic gain and exposure control (AGC)
- Automatic white balance (AWB)
- Black level correction (BLC)
- Tone mapping (Gamma)

AWB is implemented using a Greyworld algorithm, and calculates the red and blue gains to apply to generate a neutral grey frame overall.

AGC is handled by calculating a histogram of the green channel to estimate an analogue gain and exposure time which will provide a well exposed frame. A low-pass IIR filter is used to smooth the changes to the sensor to reduce perceivable steps.

The tone mapping algorithm provides a gamma correction table to improve the contrast of the scene.

The black level compensation algorithm subtracts a hardcoded black level from all pixels.

The IPU3 ImgU has further processing blocks to support image quality improvements through bayer and temporal noise reductions, however those are not supported in the current implementation, and will use default settings as provided by the kernel driver.

Demosaicing is operating with the default parameters and could be further optimised to provide improved sharpening coefficients, checker artifact removal, and false color correction.

Additional image enhancements can be made by providing lens and sensor-specific tuning to adapt for Black Level compensation (BLC), Lens shading correction (SHD) and Color correction (CCM).

## Member Function Documentation

## ◆ computeParams()

void libcamera::ipa::ipu3::IPAIPU3::computeParams | ( | const uint32_t | frame,
|  | const uint32_t | bufferId )
| override

Fill and return a buffer with ISP processing parameters for a frame.

Parameters

  [in] | frame | The frame number
  [in] | bufferId | ID of the parameter buffer to fill

Algorithms are expected to fill the IPU3 parameter buffer for the next frame given their most recent processing of the ImgU statistics.

## ◆ configure()

int libcamera::ipa::ipu3::IPAIPU3::configure | ( | const IPAConfigInfo & | configInfo,
|  | ControlInfoMap * | ipaControls )
| override

Configure the IPU3 IPA.

Parameters

  [in] | configInfo | The IPA configuration data, received from the pipeline handler
  [in] | ipaControls | The IPA controls to update

Calculate the best grid for the statistics based on the pipeline handler BDS output, and parse the minimum and maximum exposure and analogue gain control values.

Todo
  Document what the BDS is, ideally in a block diagram of the ImgU.

All algorithm modules are called to allow them to prepare the IPASessionConfiguration structure for the IPAContext.

## ◆ init()

int libcamera::ipa::ipu3::IPAIPU3::init | ( | const IPASettings & | settings,
|  | const IPACameraSensorInfo & | sensorInfo,
|  | const ControlInfoMap & | sensorControls,
|  | ControlInfoMap * | ipaControls )
| override

Initialize the IPA module and its controls.

This function receives the camera sensor information from the pipeline handler, computes the limits of the controls it handles and returns them in the ipaControls output parameter.

## ◆ logPrefix()

std::string libcamera::ipa::ipu3::IPAIPU3::logPrefix | ( |  | ) | const
| overrideprotectedvirtual

Retrieve a string to be prefixed to the log message.

This function allows classes inheriting from the Loggable class to extend the logger with an object-specific prefix output right before the log message contents.

Returns
  A string to be prefixed to the log message

Implements libcamera::Loggable.

## ◆ mapBuffers()

void libcamera::ipa::ipu3::IPAIPU3::mapBuffers | ( | const std::vector< IPABuffer > & | buffers | ) |
| override

Map the parameters and stats buffers allocated in the pipeline handler.

Parameters

  [in] | buffers | The buffers to map

## ◆ processStats()

void libcamera::ipa::ipu3::IPAIPU3::processStats | ( | const uint32_t | frame,
|  | const int64_t | frameTimestamp,
|  | const uint32_t | bufferId,
|  | const ControlList & | sensorControls )
| override

Process the statistics generated by the ImgU.

Parameters

  [in] | frame | The frame number
  [in] | frameTimestamp | Timestamp of the frame
  [in] | bufferId | ID of the statistics buffer
  [in] | sensorControls | Sensor controls

Parse the most recently processed image statistics from the ImgU. The statistics are passed to each algorithm module to run their calculations and update their state accordingly.

## ◆ queueRequest()

void libcamera::ipa::ipu3::IPAIPU3::queueRequest | ( | const uint32_t | frame,
|  | const ControlList & | controls )
| override

Queue a request and process the control list from the application.

Parameters

  [in] | frame | The number of the frame which will be processed next
  [in] | controls | The controls for the frame

Parse the request to handle any IPA-managed controls that were set from the application such as manual sensor settings.

## ◆ unmapBuffers()

void libcamera::ipa::ipu3::IPAIPU3::unmapBuffers | ( | const std::vector< unsigned int > & | ids | ) |
| override

Unmap the parameters and stats buffers.

Parameters

  [in] | ids | The IDs of the buffers to unmap

The documentation for this class was generated from the following file:

- src/ipa/ipu3/ipu3.cpp