| commit | 2984cfd224176893d335f358f69c5193cd138e50 | [log] [tgz] |
|---|---|---|
| author | Madhu Peringassery Krishnan <[email protected]> | Mon Mar 15 12:41:49 2021 -0700 |
| committer | Madhu Peringassery Krishnan <[email protected]> | Mon Mar 15 12:41:49 2021 -0700 |
| tree | e7ab451928b5e3d216bc6bf5733aceab96f42b34 | |
| parent | 0548a2fb0f39a8fc29b79f1d36f83ca807edbc62 [diff] |
SIMD updates & unit testing Change-Id: I6e35237c3045d0b421300ae6f62c3fcc9d1f7e70
The AV1 library source code is stored in the Alliance for Open Media Git repository:
$ git clone https://aomedia.googlesource.com/aom
# By default, the above command stores the source in the aom directory:
$ cd aom
CMake replaces the configure step typical of many projects. Running CMake will produce configuration and build files for the currently selected CMake generator. For most systems the default generator is Unix Makefiles. The basic form of a makefile build is the following:
$ cmake path/to/aom
$ make
The above will generate a makefile build that produces the AV1 library and applications for the current host system after the make step completes successfully. The compiler chosen varies by host platform, but a general rule applies: On systems where cc and c++ are present in $PATH at the time CMake is run the generated build will use cc and c++ by default.
The AV1 codec library has a great many configuration options. These come in two varieties:
ENABLE_FEATURE.CONFIG_FEATURE.Both types of options are set at the time CMake is run. The following example enables ccache and disables the AV1 encoder:
$ cmake path/to/aom -DENABLE_CCACHE=1 -DCONFIG_AV1_ENCODER=0
$ make
The available configuration options are too numerous to list here. Build system configuration options can be found at the top of the CMakeLists.txt file found in the root of the AV1 repository, and AV1 codec configuration options can currently be found in the file build/cmake/aom_config_defaults.cmake.
A dylib (shared object) build of the AV1 codec library can be enabled via the CMake built in variable BUILD_SHARED_LIBS:
$ cmake path/to/aom -DBUILD_SHARED_LIBS=1
$ make
This is currently only supported on non-Windows targets.
Depending on the generator used there are multiple ways of going about debugging AV1 components. For single configuration generators like the Unix Makefiles generator, setting CMAKE_BUILD_TYPE to Debug is sufficient:
$ cmake path/to/aom -DCMAKE_BUILD_TYPE=Debug
For Xcode, mainly because configuration controls for Xcode builds are buried two configuration windows deep and must be set for each subproject within the Xcode IDE individually, CMAKE_CONFIGURATION_TYPES should be set to Debug:
$ cmake path/to/aom -G Xcode -DCMAKE_CONFIGURATION_TYPES=Debug
For Visual Studio the in-IDE configuration controls should be used. Simply set the IDE project configuration to Debug to allow for stepping through the code.
In addition to the above it can sometimes be useful to debug only C and C++ code. To disable all assembly code and intrinsics set AOM_TARGET_CPU to generic at generation time:
$ cmake path/to/aom -DAOM_TARGET_CPU=generic
For the purposes of building the AV1 codec and applications and relative to the scope of this guide, all builds for architectures differing from the native host architecture will be considered cross compiles. The AV1 CMake build handles cross compiling via the use of toolchain files included in the AV1 repository. The toolchain files available at the time of this writing are:
The following example demonstrates use of the x86-macos.cmake toolchain file on a x86_64 MacOS host:
$ cmake path/to/aom \
-DCMAKE_TOOLCHAIN_FILE=path/to/aom/build/cmake/toolchains/x86-macos.cmake
$ make
To build for an unlisted target creation of a new toolchain file is the best solution. The existing toolchain files can be used a starting point for a new toolchain file since each one exposes the basic requirements for toolchain files as used in the AV1 codec build.
As a temporary work around an unoptimized AV1 configuration that builds only C and C++ sources can be produced using the following commands:
$ cmake path/to/aom -DAOM_TARGET_CPU=generic
$ make
In addition to the above it's important to note that the toolchain files suffixed with gcc behave differently than the others. These toolchain files attempt to obey the $CROSS environment variable.
Sanitizer integration is built-in to the CMake build system. To enable a sanitizer, add -DSANITIZE=<type> to the CMake command line. For example, to enable address sanitizer:
$ cmake path/to/aom -DSANITIZE=address
$ make
Sanitizers available vary by platform, target, and compiler. Consult your compiler documentation to determine which, if any, are available.
Building the AV1 codec library in Microsoft Visual Studio is supported. Visual Studio 2019 (16.7) or later is required. The following example demonstrates generating projects and a solution for the Microsoft IDE:
# This does not require a bash shell; Command Prompt (cmd.exe) is fine.
# This assumes the build host is a Windows x64 computer.
# To build with Visual Studio 2019 for the x64 target:
$ cmake path/to/aom -G "Visual Studio 16 2019"
$ cmake --build .
# To build with Visual Studio 2019 for the 32-bit x86 target:
$ cmake path/to/aom -G "Visual Studio 16 2019" -A Win32
$ cmake --build .
NOTE: The build system targets Windows 7 or later by compiling files with -D_WIN32_WINNT=0x0601.
Building the AV1 codec library in Xcode is supported. The following example demonstrates generating an Xcode project:
$ cmake path/to/aom -G Xcode
Building the AV1 codec library with Emscripten is supported. Typically this is used to hook into the AOMAnalyzer GUI application. These instructions focus on using the inspector with AOMAnalyzer, but all tools can be built with Emscripten.
It is assumed here that you have already downloaded and installed the EMSDK, installed and activated at least one toolchain, and setup your environment appropriately using the emsdk_env script.
Download AOMAnalyzer.
Configure the build:
$ cmake path/to/aom \
-DENABLE_CCACHE=1 \
-DAOM_TARGET_CPU=generic \
-DENABLE_DOCS=0 \
-DENABLE_TESTS=0 \
-DCONFIG_ACCOUNTING=1 \
-DCONFIG_INSPECTION=1 \
-DCONFIG_MULTITHREAD=0 \
-DCONFIG_RUNTIME_CPU_DETECT=0 \
-DCONFIG_WEBM_IO=0 \
-DCMAKE_TOOLCHAIN_FILE=path/to/emsdk-portable/.../Emscripten.cmake
$ make inspect
# inspect.js is in the examples sub directory of the directory in which you
# executed cmake.
$ path/to/AOMAnalyzer path/to/examples/inspect.js path/to/av1/input/file
Three variables allow for passing of additional flags to the build system.
The build system attempts to ensure the flags passed through the above variables are passed to tools last in order to allow for override of default behavior. These flags can be used, for example, to enable asserts in a release build:
$ cmake path/to/aom \
-DCMAKE_BUILD_TYPE=Release \
-DAOM_EXTRA_C_FLAGS=-UNDEBUG \
-DAOM_EXTRA_CXX_FLAGS=-UNDEBUG
After installing libvmaf.a, you can use it with the encoder:
$ cmake path/to/aom -DCONFIG_TUNE_VMAF=1
Please note that the default VMAF model (“/usr/local/share/model/vmaf_v0.6.1.pkl”) will be used unless you set the following flag when running the encoder:
# --vmaf-model-path=path/to/model
There are several methods of testing the AV1 codec. All of these methods require the presence of the AV1 source code and a working build of the AV1 library and applications.
The unit tests can be run at build time:
# Before running the make command the LIBAOM_TEST_DATA_PATH environment
# variable should be set to avoid downloading the test files to the
# cmake build configuration directory.
$ cmake path/to/aom
# Note: The AV1 CMake build creates many test targets. Running make
# with multiple jobs will speed up the test run significantly.
$ make runtests
The example tests require a bash shell and can be run in the following manner:
# See the note above about LIBAOM_TEST_DATA_PATH above.
$ cmake path/to/aom
$ make
# It's best to build the testdata target using many make jobs.
# Running it like this will verify and download (if necessary)
# one at a time, which takes a while.
$ make testdata
$ path/to/aom/test/examples.sh --bin-path examples
When making a change to the encoder run encoder tests to confirm that your change has a positive or negligible impact on encode quality. When running these tests the build configuration should be changed to enable internal encoder statistics:
$ cmake path/to/aom -DCONFIG_INTERNAL_STATS=1
$ make
The repository contains scripts intended to make running these tests as simple as possible. The following example demonstrates creating a set of baseline clips for comparison to results produced after making your change to libaom:
# This will encode all Y4M files in the current directory using the
# settings specified to create the encoder baseline statistical data:
$ cd path/to/test/inputs
# This command line assumes that run_encodes.sh, its helper script
# best_encode.sh, and the aomenc you intend to test are all within a
# directory in your PATH.
$ run_encodes.sh 200 500 50 baseline
After making your change and creating the baseline clips, you'll need to run encodes that include your change(s) to confirm that things are working as intended:
# This will encode all Y4M files in the current directory using the
# settings specified to create the statistical data for your change:
$ cd path/to/test/inputs
# This command line assumes that run_encodes.sh, its helper script
# best_encode.sh, and the aomenc you intend to test are all within a
# directory in your PATH.
$ run_encodes.sh 200 500 50 mytweak
After creating both data sets you can use test/visual_metrics.py to generate a report that can be viewed in a web browser:
$ visual_metrics.py metrics_template.html "*stt" baseline mytweak \
> mytweak.html
You can view the report by opening mytweak.html in a web browser.