Quickstart

If you're comfortable with Julia or have used another image-processing package before, this page may help you get started quickly. If some of the terms or concepts here seem strange, don't worry—there are much more detailed explanations in the following sections.

To start with, let's load the Images.jl package:

julia> using Images

Images are just arrays

For most purposes, any AbstractArray can be treated as an image. For example, numeric array can be interpreted as a grayscale image.

julia> img = rand(4, 3)4×3 Matrix{Float64}:
 0.517866   0.525796   0.374141
 0.749719   0.840641   0.728543
 0.552534   0.523948   0.00277646
 0.0834401  0.0128461  0.134685

Don't worry if you don't get the "image" result, that's expected because it's actually recognized as a numerical array and not an image. You will learn how to automatically display an image later in JuliaImages.

We could also select a region-of-interest from a larger image

# generate an image that starts black in the upper left
# and gets bright in the lower right
img = Array(reshape(range(0,stop=1,length=10^4), 100, 100))
# make a copy
img_c = img[51:70, 21:70] # see the red region
# make a view
img_v = @view img[16:35, 41:90] # see the blue region

As you might know, changing the value of a view affects the original image, while changing a copy does not:

fill!(img_c, 1) # red region in original doesn't change
fill!(img_v, 0) # blue region is changed

Some add-on packages enable additional behavior. For example,

using Unitful, AxisArrays
using Unitful: mm, s

img = AxisArray(rand(256, 256, 6, 50), (:x, :y, :z, :time), (0.4mm, 0.4mm, 1mm, 2s))

defines a 4d image (3 space dimensions plus one time dimension) with the specified name and physical pixel spacing for each coordinate. The AxisArrays package supports rich and efficient operations on such arrays, and can be useful to keep track of not just pixel spacing but the orientation convention used for multidimensional images.

JuliaImages interoperates smoothly with AxisArrays and many other packages. As further examples,

• the ImageMetadata package (incorporated into Images itself) allows you to "tag" images with custom metadata

• the IndirectArrays package supports indexed (colormap) images

• the MappedArrays package allows you to represent lazy value-transformations, facilitating work with images that may be too large to store in memory at once

• ImageTransformations allows you to encode rotations, shears, deformations, etc., either eagerly or lazily

It is very easy to define new array types in Julia–and consequently specialized images or operations–and have them interoperate smoothly with the vast majority of functions in JuliaImages.

Elements of images are pixel colorants

Elements of an image are called pixels, and JuliaImages treats pixels as first-class objects. For example, we have Gray for grayscale pixels, RGB for RGB color pixels, Lab for Lab colors, etc.

To create a pixel, initiate a struct of that type:

Gray(0.0) # black
Gray(1.0) # white
RGB(1.0, 0.0, 0.0) # red
RGB(0.0, 1.0, 0.0) # green
RGB(0.0, 0.0, 1.0) # blue

An image is just an array of pixel objects:

julia> img_gray = rand(Gray, 2, 2)2×2 Array{Gray{Float64},2} with eltype Gray{Float64}:
 Gray{Float64}(0.412225)   Gray{Float64}(0.755344)
 Gray{Float64}(0.0368978)  Gray{Float64}(0.198098)
julia> img_rgb = rand(RGB, 2, 2)2×2 Array{RGB{Float64},2} with eltype RGB{Float64}:
 RGB{Float64}(0.709158,0.584705,0.186682)  …  RGB{Float64}(0.202959,0.435521,0.116792)
 RGB{Float64}(0.760013,0.693292,0.20883)      RGB{Float64}(0.0709577,0.888894,0.310744)
julia> img_lab = rand(Lab, 2, 2)2×2 Array{Lab{Float64},2} with eltype Lab{Float64}:
 Lab{Float64}(14.7804,-55.7728,-50.3997)  …  Lab{Float64}(68.5718,-91.6112,127.303)
 Lab{Float64}(61.1431,62.8829,-55.5891)      Lab{Float64}(74.6444,-105.937,81.6293)

Color channels are not their own dimension of the image array; rather, an image is an array of pixels. As you can see, both img_rgb and img_lab images are of size $2 \times 2$ (instead of $2 \times 2 \times 3$ or $3 \times 2 \times 2$).

This design choice facilitates generic code that can handle both grayscale and color images without needing to introduce extra loops or checks for a color dimension. It also provides more rational support for 3d grayscale images—which might happen to have size 3 along the third dimension—and consequently helps unify the "computer vision" and "biomedical image processing" communities.

Converting between colorants and separate color channels

Conversions between different Colorants are straightforward:

julia> RGB.(img_gray) # Gray => RGB2×2 Array{RGB{Float64},2} with eltype RGB{Float64}:
 RGB{Float64}(0.412225,0.412225,0.412225)     …  RGB{Float64}(0.755344,0.755344,0.755344)
 RGB{Float64}(0.0368978,0.0368978,0.0368978)     RGB{Float64}(0.198098,0.198098,0.198098)
julia> Gray.(img_rgb) # RGB => Gray2×2 Array{Gray{Float64},2} with eltype Gray{Float64}:
 Gray{Float64}(0.576542)  Gray{Float64}(0.32965)
 Gray{Float64}(0.658013)  Gray{Float64}(0.578422)

Sometimes, to work with other packages, you'll need to convert a $m \times n$ RGB image to $m \times n \times 3$ numeric array and vice versa. The functions channelview and colorview are designed for this purpose. For example:

julia> img_CHW = channelview(img_rgb) # 3 * 2 * 23×2×2 reinterpret(reshape, Float64, ::Array{RGB{Float64},2}) with eltype Float64:
[:, :, 1] =
 0.709158  0.760013
 0.584705  0.693292
 0.186682  0.20883

[:, :, 2] =
 0.202959  0.0709577
 0.435521  0.888894
 0.116792  0.310744
julia> img_HWC = permutedims(img_CHW, (2, 3, 1)) # 2 * 2 * 32×2×3 Array{Float64, 3}:
[:, :, 1] =
 0.709158  0.202959
 0.760013  0.0709577

[:, :, 2] =
 0.584705  0.435521
 0.693292  0.888894

[:, :, 3] =
 0.186682  0.116792
 0.20883   0.310744

To convert a channelview array into an array of pixels, ensure the first dimension is the color channel and use colorview:

julia> img_rgb = colorview(RGB, img_CHW) # 2 * 22×2 Array{RGB{Float64},2} with eltype RGB{Float64}:
 RGB{Float64}(0.709158,0.584705,0.186682)  …  RGB{Float64}(0.202959,0.435521,0.116792)
 RGB{Float64}(0.760013,0.693292,0.20883)      RGB{Float64}(0.0709577,0.888894,0.310744)

For Gray images, the following codes are almost equivalent except that the construction version copies the data while the view version doesn't.

img_num = rand(4, 4)

img_gray_copy = Gray.(img_num) # construction
img_num_copy = Float64.(img_gray_copy) # construction

img_gray_view = colorview(Gray, img_num) # view
img_num_view = channelview(img_gray_view) # view

The 0-to-1 intensity scale

In JuliaImages, images are displayed assuming by default that 0 means "black" and 1 means "white" (or "saturated" for an RGB image).

Importantly, this applies even for intensities encoded as 8-bit integers which themselves range from 0 to 255 (not from 0 to 1). This is made possible with the special number type N0f8 which interprets an 8-bit "integer" as if it had been scaled by 1/255, encoding values from 0 to 1 in 256 steps.

The name N0f8 stands for Normalized, with 0 integer bits and 8 fractional bits. There are other types like N0f16 for working with 16-bit images (and even N2f14 for images acquired with a 14-bit camera, etc.).

julia> img_n0f8 = rand(N0f8, 2, 2)2×2 reinterpret(N0f8, ::Matrix{UInt8}):
 0.435  0.725
 0.816  0.192
julia> float.(img_n0f8)2×2 Matrix{Float32}:
 0.435294  0.72549
 0.815686  0.192157

Although it's not recommended, but you can use rawview to get the underlying storage data and convert it to UInt8 (or other types) if you insist.

julia> img_n0f8_raw = rawview(img_n0f8)2×2 rawview(reinterpret(N0f8, ::Matrix{UInt8})) with eltype UInt8:
 0x6f  0xb9
 0xd0  0x31
julia> float.(img_n0f8_raw)2×2 Matrix{Float64}:
 111.0  185.0
 208.0   49.0

Conversions between the storage type, i.e., the actual numeric type, without changing the color type are supported by the following functions:

• float32, float64
• n0f8, n6f10, n4f12, n2f14, n0f16

julia> img = rand(Gray{N0f8}, 2, 2)2×2 Array{Gray{N0f8},2} with eltype Gray{N0f8}:
 Gray{N0f8}(0.925)  Gray{N0f8}(0.267)
 Gray{N0f8}(0.345)  Gray{N0f8}(0.827)
julia> img_float32 = float32.(img) # Gray{N0f8} => Gray{Float32}2×2 Array{Gray{Float32},2} with eltype Gray{Float32}:
 Gray{Float32}(0.92549)   Gray{Float32}(0.266667)
 Gray{Float32}(0.345098)  Gray{Float32}(0.827451)
julia> img_n0f16 = n0f16.(img_float32) # Gray{Float32} => Gray{N0f16}2×2 Array{Gray{N0f16},2} with eltype Gray{N0f16}:
 Gray{N0f16}(0.92549)  Gray{N0f16}(0.26667)
 Gray{N0f16}(0.3451)   Gray{N0f16}(0.82745)

If you don't want to specify the destination type, float is designed for this:

julia> img_n0f8 = rand(Gray{N0f8}, 2, 2)2×2 Array{Gray{N0f8},2} with eltype Gray{N0f8}:
 Gray{N0f8}(0.886)  Gray{N0f8}(0.98)
 Gray{N0f8}(0.522)  Gray{N0f8}(0.412)
julia> img_float = float.(img_n0f8) # Gray{N0f8} => Gray{Float32}2×2 Array{Gray{Float32},2} with eltype Gray{Float32}:
 Gray{Float32}(0.886275)  Gray{Float32}(0.980392)
 Gray{Float32}(0.521569)  Gray{Float32}(0.411765)

For a view-like conversion without new memory allocation, of_eltype in MappedArrays is designed for this:

julia> using MappedArrays
julia> img_float_view = of_eltype(Gray{Float32}, img_n0f8)2×2 mappedarray(MappedArrays.var"#7#9"{Gray{Float32}}(), MappedArrays.var"#8#10"{Gray{N0f8}}(), ::Array{Gray{N0f8},2}) with eltype Gray{Float32}:
 Gray{Float32}(0.886275)  Gray{Float32}(0.980392)
 Gray{Float32}(0.521569)  Gray{Float32}(0.411765)
julia> eltype(img_float_view)Gray{Float32}

Arrays with arbitrary indices

If you have an input image and perform some kind of spatial transformation on it, how do pixels (or voxels) in the transformed image match up to pixels in the input? Through Julia's support for arrays with indices that start at values other than 1, it is possible to allow array indices to represent absolute position in space, making it straightforward to keep track of the correspondence between location across multiple images. More information can be found in Keeping track of location with unconventional indices.

Index of functions

See Summary and function reference for more information about each of the topics below. The list below is accessible via ?Images from the Julia REPL. If you've used other frameworks previously, you may also be interested in the Comparison with other image processing frameworks. Also described are the ImageFeatures.jl and ImageSegmentation.jl packages, which support a number of algorithms important for computer vision.

Constructors, conversions, and traits:

• Construction: use constructors of specialized packages, e.g., AxisArray, ImageMeta, etc.
• "Conversion": colorview, channelview, rawview, normedview, PermutedDimsArray, paddedviews
• Traits: pixelspacing, sdims, timeaxis, timedim, spacedirections

Contrast/coloration:

• clamp01, clamp01nan, scaleminmax, colorsigned, scalesigned

Algorithms:

• Reductions: maxfinite, maxabsfinite, minfinite, meanfinite, sad, ssd, integral_image, boxdiff, gaussian_pyramid
• Resizing and spatial transformations: restrict, imresize, warp
• Filtering: imfilter, imfilter!, imfilter_LoG, mapwindow, imROF, padarray
• Filtering kernels: Kernel. or KernelFactors., followed by ando[345], guassian2d, imaverage, imdog, imlaplacian, prewitt, sobel
• Exposure : build_histogram, adjust_histogram, imadjustintensity, imstretch, imcomplement, AdaptiveEqualization, GammaCorrection, cliphist
• Gradients: backdiffx, backdiffy, forwarddiffx, forwarddiffy, imgradients
• Edge detection: imedge, imgradients, thin_edges, magnitude, phase, magnitudephase, orientation, canny
• Corner detection: imcorner, harris, shi_tomasi, kitchen_rosenfeld, meancovs, gammacovs, fastcorners
• Blob detection: blob_LoG, findlocalmaxima, findlocalminima
• Morphological operations: dilate, erode, closing, opening, tophat, bothat, morphogradient, morpholaplace, feature_transform, distance_transform
• Connected components: label_components, component_boxes, component_lengths, component_indices, component_subscripts, component_centroids
• Interpolation: bilinear_interpolation

Test images and phantoms (see also TestImages.jl):

• shepp_logan