Quickstart

Quickstart

If you're comfortable with Julia or have used another image-processing package before, this page may help you get started quickly. If some of the terms or concepts here seem strange, don't worry–-there are much more detailed explanations in the following sections.

Images are just arrays

For most purposes, any AbstractArray can be treated as an image. For example, numeric array can be interpreted as a grayscale image.

julia> img = rand(4, 4)
4×4 Array{Float64,2}:
 0.445251  0.0130485  0.473196   0.654294
 0.487617  0.163802   0.974613   0.772516
 0.312217  0.665554   0.0494935  0.868558
 0.461896  0.454225   0.502677   0.706888

We could also select a region-of-interest from a larger image

# generate an image that starts black in the upper left
# and gets bright in the lower right
img = Array(reshape(range(0,stop=1,length=10^4), 100, 100))
# make a copy
img_c = img[51:70, 21:70] # red
# make a view
img_v = @view img[16:35, 41:90] # blue

As you might know, changing the value of a view would affect the original image, while changing that of a copy doesn't:

fill!(img_c, 1) # red region in original doesn't change
fill!(img_v, 0) # blue

Don't worry if you don't get the "image" result, that's expected and you'll learn how to automatically display an image later in JuliaImages.

Some add-on packages enable additional behavior. For example,

using Unitful, AxisArrays
using Unitful: mm, s

img = AxisArray(rand(256, 256, 6, 50), (:x, :y, :z, :time), (0.4mm, 0.4mm, 1mm, 2s))

defines a 4d image (3 space dimensions plus one time dimension) with the specified name and physical pixel spacing for each coordinate. The AxisArrays package supports rich and efficient operations on such arrays, and can be useful to keep track of not just pixel spacing but the orientation convention used for multidimensional images.

JuliaImages interoperates smoothly with AxisArrays and many other packages. As further examples,

It is very easy to define new array types in Julia–and consequently specialized images or operations–and have them interoperate smoothly with the vast majority of functions in JuliaImages.

Array elements are pixels (and vice versa)

Elements of image are called pixels; in JuliaImages we provide an abstraction on this concept. For example, we have Gray for grayscale image, RGB for RGB image, Lab for Lab image, and etc.

Creating a pixel is initializing a struct of that type:

Gray(0.0) # black
Gray(1.0) # white
RGB(1.0, 0.0, 0.0) # red
RGB(0.0, 1.0, 0.0) # green
RGB(0.0, 0.0, 1.0) # blue

and image is just an array of pixel objects:

julia> img_gray = rand(Gray, 2, 2)
2×2 Array{Gray{Float64},2} with eltype Gray{Float64}:
 Gray{Float64}(0.35863)    Gray{Float64}(0.312923)
 Gray{Float64}(0.0374148)  Gray{Float64}(0.560905)

julia> img_rgb = rand(RGB, 2, 2)
2×2 Array{RGB{Float64},2} with eltype RGB{Float64}:
 RGB{Float64}(0.793579,0.595972,0.8822)    …  RGB{Float64}(0.328106,0.666879,0.120543)
 RGB{Float64}(0.441217,0.376236,0.151228)     RGB{Float64}(0.554866,0.326394,0.0658588)

julia> img_lab = rand(Lab, 2, 2)
2×2 Array{Lab{Float64},2} with eltype Lab{Float64}:
 Lab{Float64}(6.424,108.233,-38.6783)   …  Lab{Float64}(38.2976,-22.9085,54.2427)
 Lab{Float64}(80.5625,-71.7646,-86.06)     Lab{Float64}(3.13083,3.30244,78.7737)

As you can see, both img_rgb and img_lab images are of size $2 \times 2$ (instead of $2 \times 2 \times 3$ or $3 \times 2 \times 2$); a RGB image is an array of RGB pixels whereas a Lab image is an array of Lab pixel.

Note

It's recommended to use Gray instead of the Number type in JuliaImages since it indicates that the array of numbers is best interpreted as a grayscale image. For example, it triggers Atom/Juno and Jupyter to display the array as an image instead of a matrix of numbers. There's no performance overhead for using Gray over Number.

This design choice facilitates generic code that can handle both grayscale and color images without needing to introduce extra loops or checks for a color dimension. It also provides more rational support for 3d grayscale images–which might happen to have size 3 along the third dimension–and consequently helps unify the "computer vision" and "biomedical image processing" communities.

Color conversions are construction/view

Conversions between different Colorants are straightforward:

julia> RGB.(img_gray) # Gray => RGB
2×2 Array{RGB{Float64},2} with eltype RGB{Float64}:
 RGB{Float64}(0.35863,0.35863,0.35863)        …  RGB{Float64}(0.312923,0.312923,0.312923)
 RGB{Float64}(0.0374148,0.0374148,0.0374148)     RGB{Float64}(0.560905,0.560905,0.560905)

julia> Gray.(img_rgb) # RGB => Gray
2×2 Array{Gray{Float64},2} with eltype Gray{Float64}:
 Gray{Float64}(0.687687)  Gray{Float64}(0.503303)
 Gray{Float64}(0.370014)  Gray{Float64}(0.365006)
Note

You'll see broadcasting semantics used in JuliaImages here and there, check the documentation if you're not familiar with it.

Sometimes, to work with other packages, you'll need to convert a $m \times n$ RGB image to $m \times n \times 3$ numeric array and vice versa. The functions channelview and colorview are designed for this purpose. For example:

julia> img_CHW = channelview(img_rgb) # 3 * 2 * 2
3×2×2 reinterpret(Float64, ::Array{RGB{Float64},3}):
[:, :, 1] =
 0.793579  0.441217
 0.595972  0.376236
 0.8822    0.151228

[:, :, 2] =
 0.328106  0.554866
 0.666879  0.326394
 0.120543  0.0658588

julia> img_HWC = permutedims(img_CHW, (2, 3, 1)) # 2 * 2 * 3
2×2×3 Array{Float64,3}:
[:, :, 1] =
 0.793579  0.328106
 0.441217  0.554866

[:, :, 2] =
 0.595972  0.666879
 0.376236  0.326394

[:, :, 3] =
 0.8822    0.120543
 0.151228  0.0658588
julia> img_CHW = permutedims(img_HWC, (3, 1, 2)) # 3 * 2 * 2
3×2×2 Array{Float64,3}:
[:, :, 1] =
 0.793579  0.441217
 0.595972  0.376236
 0.8822    0.151228

[:, :, 2] =
 0.328106  0.554866
 0.666879  0.326394
 0.120543  0.0658588

julia> img_rgb = colorview(RGB, img_CHW) # 2 * 2
2×2 reshape(reinterpret(RGB{Float64}, ::Array{Float64,3}), 2, 2) with eltype RGB{Float64}:
 RGB{Float64}(0.793579,0.595972,0.8822)    …  RGB{Float64}(0.328106,0.666879,0.120543)
 RGB{Float64}(0.441217,0.376236,0.151228)     RGB{Float64}(0.554866,0.326394,0.0658588)
Warning

Don't overuse channelview because it loses the colorant information by converting an image to a raw numeric array.

It's very likely that users from other languages will have the tendency to channelview every image they're going to process. Unfamiliarity of the pixel concept provided by JuliaImages doesn't necessarily mean it's bad.

Note

The reason we use CHW (i.e., channel-height-width) order instead of HWC is that this provides a memory friendly indexing mechanisim for Array. By default, in Julia the first index is also the fastest (i.e., has adjacent storage in memory). For more details, please refer to the performance tip: Access arrays in memory order, along columns

You can use permuteddimsview to "reinterpret" the orientation of a chunk of memory without making a copy, or permutedims if you want a copy.

For Gray images, the following codes are almost equivalent except that the construction version copies the data while the view version doesn't.

img_num = rand(4, 4)

img_gray_copy = Gray.(img_num) # construction
img_num_copy = Float64.(img_gray_copy) # construction

img_gray_view = colorview(Gray, img_num) # view
img_num_view = channelview(img_gray_view) # view

The 0-to-1 intensity scale

In JuliaImages, by default all images are displayed assuming that 0 means "black" and 1 means "white" or "saturated" (the latter applying to channels of an RGB image).

Perhaps surprisingly, this 0-to-1 convention applies even when the intensities are encoded using only 8-bits per color channel. JuliaImages uses a special type, N0f8, that interprets an 8-bit "integer" as if it had been scaled by 1/255, thus encoding values from 0 to 1 in 256 steps.

N0f8 numbers (standing for Normalized, with 0 integer bits and 8 fractional bits) obey standard mathematical rules, and can be added, multiplied, etc. There are types like N0f16 for working with 16-bit images (and even N2f14 for images acquired with a 14-bit camera, etc.).

julia> img_n0f8 = rand(N0f8, 2, 2)
2×2 reinterpret(N0f8, ::Array{UInt8,2}):
 0.98   0.965
 0.812  0.024

julia> float.(img_n0f8)
2×2 Array{Float32,2}:
 0.980392  0.964706
 0.811765  0.0235294
Note

This infrastructure allows us to unify "integer" and floating-point images, and avoids the need for special conversion functions (e.g., im2double in MATLAB) that change the value of pixels when your main goal is simply to change the type (numeric precision and properties) used to represent the pixel.

Although it's not recommended, but you can use rawview to get the underlying storage data and convert it to UInt8 (or other types) if you insist.

julia> img_n0f8_raw = rawview(img_n0f8)
2×2 rawview(reinterpret(N0f8, ::Array{UInt8,2})) with eltype UInt8:
 0xfa  0xf6
 0xcf  0x06

julia> float.(img_n0f8_raw)
2×2 Array{Float64,2}:
 250.0  246.0
 207.0    6.0

Conversions between the storage type without changing the color type are supported by the following functions:

julia> img = rand(Gray{N0f8}, 2, 2)
2×2 Array{Gray{N0f8},2} with eltype Gray{Normed{UInt8,8}}:
 Gray{N0f8}(0.482)  Gray{N0f8}(0.518)
 Gray{N0f8}(0.435)  Gray{N0f8}(0.231)

julia> img_float32 = float32.(img) # Gray{N0f8} => Gray{Float32}
2×2 Array{Gray{Float32},2} with eltype Gray{Float32}:
 Gray{Float32}(0.482353)  Gray{Float32}(0.517647)
 Gray{Float32}(0.435294)  Gray{Float32}(0.231373)

julia> img_n0f16 = n0f16.(img_float32) # Gray{Float32} => Gray{N0f16}
2×2 Array{Gray{N0f16},2} with eltype Gray{Normed{UInt16,16}}:
 Gray{N0f16}(0.48235)  Gray{N0f16}(0.51765)
 Gray{N0f16}(0.43529)  Gray{N0f16}(0.23137)

If you don't want to specify the destination type, float is designed for this:

julia> img_n0f8 = rand(Gray{N0f8}, 2, 2)
2×2 Array{Gray{N0f8},2} with eltype Gray{Normed{UInt8,8}}:
 Gray{N0f8}(0.78)   Gray{N0f8}(0.953)
 Gray{N0f8}(0.365)  Gray{N0f8}(0.475)

julia> img_float = float.(img_n0f8) # Gray{N0f8} => Gray{Float32}
2×2 Array{Gray{Float32},2} with eltype Gray{Float32}:
 Gray{Float32}(0.780392)  Gray{Float32}(0.952941)
 Gray{Float32}(0.364706)  Gray{Float32}(0.47451)

For a view-like conversion without new memory allocation, of_eltype in MappedArrays is designed for this:

julia> using MappedArrays

julia> img_float_view = of_eltype(Gray{Float32}, img_n0f8)
2×2 mappedarray(x->(MappedArrays.convert)($(Expr(:static_parameter, 1)), x), y->(MappedArrays.convert)($(Expr(:static_parameter, 1)), y), ::Array{Gray{N0f8},2}) with eltype Gray{Float32}:
 Gray{Float32}(0.780392)  Gray{Float32}(0.952941)
 Gray{Float32}(0.364706)  Gray{Float32}(0.47451)

julia> eltype(img_float_view)
Gray{Float32}

Arrays with arbitrary indices

If you have an input image and perform some kind of spatial transformation on it, how do pixels/voxels in the transformed image match up to pixels in the input? Through Julia's support for arrays with indices that start at values other than 1, it is possible to allow array indices to represent absolute position in space, making it straightforward to keep track of the correspondence between location across multiple images. More information can be found in Keeping track of location with unconventional indices.

Display

Currently there're five julia packages can be used to display an image:

Warning

Currently ImageView is not fully tested on MacOS and Windows; check ImageView#146 and ImageView#175 to get an update.

Examples of usage

If you feel ready to get started, see the Demonstrations page for inspiration.

Function categories

See Summary and function reference for more information about each of the topics below. The list below is accessible via ?Images from the Julia REPL. If you've used other frameworks previously, you may also be interested in the Comparison with other image processing frameworks. Also described are the ImageFeatures.jl and ImageSegmentation.jl packages, which support a number of algorithms important for computer vision.

Constructors, conversions, and traits:

Contrast/coloration:

Algorithms:

Test images and phantoms (see also TestImages.jl):