Pliant graphical stack image layer

Table of content

This is a fairly long article, so here is a short summary of what you will find in it:

Overview

The image layer interface is defined in module /pliant/graphic/image/prototype.pli

Providing the characteristics of the image

The simplest way of defining an image prototype is:

image_prototype 10 20 30 40 100 200 color_gamut:"rgb"

The first four parameters of image_prototype function are the left top right and bottom coordinates of the borders of the images, in millimeters.
Then, 100 and 200 are the number of columns and rows in the image, and the last parameter is the last parameter using 'color_gamut' function describes selected color model for the image.

Another way is to provide the image resolution rather than the number of pixels:

image_prototype 10 20 30 40 300 600 4 4 image_adjust_extend color_gamut:"rgb"

Here, 300 and 600 are the resolution in dpi (dot per inch), and the next to come double 4 specifies that the final number of columns and rows have to be a multiple of 4 (this can be important for some applications that do anti-aliasing, so group tiles of pixels in the end).
Then, image_adjust_extend means that if at the specified resolution the provided dimensions don't match an exact number of columns or rows, then the dimensions (right end bottom borders coordinates) will be extended.

Various exact dimension versus resolution adjustments policies are:

A third, less frequently used method is to derive the new image prototype from an existing one. Assuming that 'p' is an existing image, we can get an image with the (roughly) same dimensions and pixels encoding, but at 300 dpi, through:

image_prototype p "resolution 300"

The various instructions that can be used to modify the image are 'margin' 'x0' 'y0' 'x1' 'y1' 'drop_transparency' 'size_x' 'size_y' 'resolution' (with one or two parameters) and 'antialiasing' (also with one or two parameters).
See code at the end of /pliant/graphic/image/prototype.pli module for extra details.

Basic image usage

var Link:ImagePrototype img :> new ImagePixmap
img setup (image_prototype 0 0 10 10 256 256 color_gamut:"rgb") ""
for (var Int y) 0 img:size_y
  for (var Int x) 0 img:size_x
    var ColorRGB888 c := color rgb x y 0
    img write x y 1 addressof:c
img read 100 200 1 addressof:c
console "red value is " (cast c:r Int) eol

'setup' is used to resize the image pixels grid according to the provided image prototype specifications. The last parameter of 'setup', here an empty string is used to provide extra configuration parameters, and we will see it later.
'size_x' and 'size_y' provide the number of columns and rows in the image.
Then, 'read' and 'write' are the functions enabling to read or write a set of consecutive pixels in the image to and from a buffer.

Please, first notice that an image must be a Pliant object, not a local variable, so writing something like:

var ImagePixmap img # Don't do that
img setup ...

instead of:

var Link:ImagePixmap img :> new ImagePixmap
img setup ...

would be a bug.

Then, please also notice that reading pixels outside image boundary is a bug that will produce unpredictable result and is likely to just crash Pliant process. When issuing:

var Int x y count
...
img read x y count

The following assertion must be satisfied: x>=0 and x+count<=img:size_x and y>=0 and y<img:size_y and count>=0

If and only if the image has type 'ImagePixmap', then we can access pixels directly using 'pixel' method that returns the address of the specified pixel:

c := (img pixel 100 200) map ColorRGB888

which is the same as:

img read 100 200 1 addressof:c

Now, informations about the color model:

console img:gamut:pixel_size eol

See color models for extra informations about color encoding provided by a gamut.

One of these, the pixel size, is a so frequently used that it is copied inside the image header in order to be accessible without the gamut indirection:

console img:pixel_size eol

Lastly, just like for color models, images have a 'configure' and 'query' method that can be used to set of read special parameters of the image. The semantic of this example will be explained later:

img configure "shrink"

Advanced image usage

Now, the huge super fantastic incredible terrific ... two cents trick about image processing is the fact that 'read' and 'write' methods don't work on a single pixel, but on several pixels at once. The reason is that if 'read' was working on a single pixel, the glue code for connection the effective function, then finding the address of the pixel in memory would be very expensive in the end since applied to millions of pixels.
On the other hand, we could read or write tiles at onces (several pixels on several lines) but it does not worth the extra complexity in most situation since the glue code cost has already been dropped to negligible through the huge super fantastic incredible terrific trick (tm).

So, pushing the trick further, we introduce an even more efficient way to access pixels, that also avoid the copy implied by the 'read' and 'write'.

for (var Int y) 0 img:size_y
  var Int x := 0
  while x<img:size_x
    var Address a := img write_map x y 1 img:size_x-x (var Int count)
    if a<>null
      for (var Int i) 0 count-1
        a map ColorRGB888 i := color rgb x+count y 0
      img write_unmap x y count adr
      x += count
    else
      var ColorRGB888 c := color rgb x y 0
      img write x y 1 addressof:c

The parameters provided to 'write_map' are the position of the first pixel, just like with 'write', then the minimum and maximum number of pixels to write, and finally a local variable that will be set on return with the effective number of pixels mapped.
On return, either mapping succeeded, so 'count' is set and the returned address is not null, or mapping failed (because it's not supported or number of pixels to map specified by minimum and maximum could not be met) and the returned address is null.
When pixels have been written, 'write_unmap' must be called to release the access to pixels.

For not yet stunned people, there is also 'read_map' and 'read_unmap'.

Then we have fill for writing the same pixels several times (here 10 times):

var ColorRGB888 c := color rgb 128 128 128
img fill 100 200 10 addressof:c

When pasting an image in another one, with a rotation transformation, being able to read and write pixels only horizontally is a serious problem, so for this situation, 2 dimensions reading is also provided:

var Address a := img rectangle_read_map 100 200 (var Int x0) (var Int y0) (var Int x1) (var Int y1) (var Int step_x) (var Int step_y)
...
img rectangle_read_unmap x0 y0 x1 y1 adr

The two first parameters in 'rectangle_read_map' provide the point that we want to read.
Then, on return 'x0', 'y0', 'x1', 'y1' contain the coordinates specifying the area that can now be accessed directly and 'adr' is the address of the pixel with coordinates (x0,y0).
So, for any (x,y) pixel that satisfies x>=x0 and x<x1 and y>=y0 and y<y1
the address of the point is (a translate Byte (x-x0)*step_x+(y-y0)*step_y)
Please notice that we have x>=x0 but x<x1 and also y>=y0 but y<y1.

In very few words, 'rectangle_read_map' says: I want to read this pixel; and the answer of it is: you can also read at once all the surrounding pixels in the provided range.

Now, about implementation, when you use an ImagePixmap uncompressed image, the image storage is not allocated as a single chunk because it might too easily overflow the address space on a 32 bits system, and also not allocated line by line because 'rectangle_read_map' would not work efficiently enough. The storage is allocated as tiles of several lines. The default size of a tile is 64 KB, but it can be adjusted through using 'tile_size' option in 'setup' method options parameter (or 'tile_y' to directly force the number of lines per tile). See 'In memory compressed images' paragraph bellow for examples of options passed to 'setup' method.

In memory compressed images

In order to ask Pliant to compress the image in memory, just use:

var Link:ImagePrototype img :> new ImagePacked

instead of:

var Link:ImagePrototype img :> new ImagePixmap

An introduction to PACK4 encoding

Let's talk a little bit about PACK4 encoding.

It's implemented in /pliant/util/encoding/pack4.pli and is an improvement of packbits encoding (see Google for an introduction to packbits), working on pixels instead of bytes (I don't know who is the idiot that decided packbits would work on bytes as opposed to pixels) and containing four instructions instead of 2:

The third code is optimizing oversampled images.
The fourth code is optimizing text on a fixed color ground through automatically creating a kind of two colors palette that avoid the color to be provided with each letter to ground and ground to letter transition.
The encoding on 'n' is also different than in packbits. It's implemented in 'read_int' and 'write_int'. See the code for extra details.

Now, the nice properties of PACK4 are:
When ripping a text+image page, at let's say 2880 dpi in order to feed a printer:

And the bad properties are:

Packed image implementation

Now, if you read the code in /pliant/graphic/image/packed.pli, you will see that it's not completely trivial, as opposed to the code handling uncompressed images, and the reason is that the code provides two features:

Let's explain all this with greater details.

The general idea is that a packed image is encoded as a set of tiles (each tile, excepts the ones on the border, contains tile_x columns and tile_y rows).
Some tiles are stored compressed, some are stored uncompressed (and some are stored both), and some are stored in an on disk temporary file.

Each time access to a tile is requested due to methods 'read' or 'write' being called, the tile is automatically uncompressed if it was compressed.
But, if too many tiles are already uncompressed (clear_size > clear_limit) then the oldest uncompressed tile will be recompressed. This is implemented in 'tile_map' method. 'tile_map' will call 'pack' and 'unpack' methods for doing the real compression and decompression of the tile.
Moreover, if the compressed content is too large (packed_size > packed_limit) then tiles will be swapped out to a temporary file.

Packed image specific features

Here are some extra features available only for in memory packed image:

First, you can read to content or write the content of a packed image to disk through:

var ExtendedStatus status := img fast_save "file:/tmp/test.packed"
if status=failure
  console "saving image failed (" status:message ")" eol

and

var ExtendedStatus satus := img fast_load "file:/tmp/test.pack"

The difference between 'fast_save' and 'fast_load' and the standard 'load' and 'save' that we will see later is that there is no uncompression then recompression: titles are written directly to the disk so it can be significantly faster is some situations.

img configure "shrink"

will force all tiles of the image to the compressed only version. It can be useful to save memory if you know that the image content will probably not be used anymore for some time,
and:

img configure "disk_shrink"

will force all compressed tiles to a temporary file.

I will end this introduction to in memory packed images through explaining configuration options.
Here is a first example:

var Link:ImagePrototype img :> new ImagePacked
img setup (image_prototype 0 0 10 10 256 256 color_gamut:"rgb") "tile_x 64 tile_y 64"

that forces the tiles size to 64 x 64 pixels instead of being automatically computed.
Tiles size can have a tiny influence on the compression efficiency, but more on time spent switching between the clear and compressed encoding for each tile, and also when 'rectangle_read_map' is used.
Then:

img setup (image_prototype 0 0 10 10 256 256 color_gamut:"rgb") "clear_cache_size "+(string 4*2^20)

would force the memory used by clear tiles to 4 MB instead of the default, and:

img setup (image_prototype 0 0 10 10 256 256 color_gamut:"rgb") "packed_cache_size "+(string 16*2^20)

would force the packed tiles to use no more than 16 MB or be swapped out to a temporary file.
Please notice that 'clear_cache_size' is automatically computed and you should not change it unless you know what you do, and on the other hand 'packed_cache_size' defaults to never swap content to the disk.
Of course, all configuration parameters can be used at once provided they are space separated.

Read filtering images

We have seen that Pliant image library supports two kind of in memory images: uncompressed and PACK4 compressed ones.
There are many more kind of images, that are filtering images. I mean, when the image content is red or written, an operation is applied on the underlying image.

ImageConvert: changing the color model

module "/pliant/graphic/image/convert.pli"
...
var Link:ImagePrototype img
...
var Link:ImageConvert c :> new ImageConvert
var ExtendedStatus s := c bind img color_gamut:"grey" ""
if s=failure
  console "cannot build the conversion filtering image: " s:message eol

There is an alternate way to achive the same, but relies on the generic 'setup' method:

var Link:ImagePrototype c :> new ImageConvert
var ExtendedStatus s := c setup img "gamut [dq]grey[dq]"
if s=failure
  console "cannot build the conversion filtering image: " s:message eol

Please notice that when 'setup' generic method is used as opposed to 'bind', then 'c' variable can either have type Link:ImagePrototype or Link:ImageConvert. The same will apply to other filters bellow

- add a fiew optimising options since some are absolutely necessary for some applications -

ImageAntialiasing

module "/pliant/graphic/image/antialiasing.pli"
...
var Link:ImageAntiAliasing aa :> new ImageAntiAliasing
var ExtendedStatus s := aa bind img 4 4

or

var ExtendedStatus s := aa setup img "antialiasing 4 4"

There are several techniques for providing proper (anti-aliased) vector drawing, text rendering, image rescaling.
Pliant mostly relies on naive algorithms applied on a 4x4 oversampled image, then downscaled using anti-aliasing. This proved to be a competitive solution as opposed to advanced algorithms working directly at final resolution.
- a full explanation would be out of the scope of this article -
The only exception is fast rendering in the UI where only text is anti-aliased directly in the font cache.

Anti-aliasing code assumes (and checks) that the color model is 8 bits per component.

ImageSharpening

module "/pliant/graphic/image/sharpening.pli"
...
var Link:ImageAntiSharpening sharp :> new ImageSharpening
var ExtendedStatus s := sharp bind img 0.2

or

var ExtendedStatus s := sharp setup img "intensity 0.2"

Sharpening level is ranging from 1 to -1 (not exactly one since more that one is still possible). 0 means neutral and negative value means blur.

The sharpening algorithm is very naive, and just relies on a 3x3 matrix with the  following coefficients:
-2  -3 -2
-3 20 -3
-2  -3 -2

On the other hand, implementation is not that naive since it does dithering in order to avoid quantization artifacts.

Sharpening code assumes (and checks) that the color model is 8 bits per component.

ImageResampling

This is superseded by ImageTransfrom.

module "/pliant/graphic/image/resampling.pli"
...
var Link:ImageResampling r :> new ImageResampling
var ExtendedStatus s := r bind img 10 10 30 30 100 100 5 5

or

var ExtendedStatus s := r setup img "area 10 10 30 30 size 100 100 translate 5 5"

The first four parameters of 'bind' method specify the coordinates of the new images borders in millimeters. Then the two next parameters specify how many columns and rows the new image will have.
The last two parameters, here two times 5, specify a translation (expressed in millimeters) to be applied to the initial image.
When using the 'setup' alternative, 'area' is optional and defaults to the same border coordinates as in the initial image, and 'translate' is also optional and defaults to 0 0.

Let's take a few examples:

var Link:ImagePrototype img :> new ImagePixmap
img setup (image_prototype 0 0 100 100 400 400 color_gamut:"rgb") ""
...
var Link:ImageResampling r :> new ImageResampling
var ExtendedStatus s := r bind img 10 10 30 30 100 100 0 0

The initial 'img' image is 100 x 100 mm, and contains 400 x 400 pixels.
The resampled 'r' image is the 20 x 20 mm area starting at 10 millimeters from both the left and the top of the initial image, and resampled to have 100 x 100 pixels.
The coordinates in millimeters of the new image top left corner are (10,10).

On the other hand, if we write:

var ExtendedStatus s := r bind img 0 0 20 20 100 100 10 10

then the pixels in the 'r' image will be the same as in the previous sample, but the coordinates in millimeters of the new image top left corner will be (0,0).

ImageRotate

This is superseded by ImageTransform.
It it intended to provide a view of the image rotated by an exact multiple of 90°:

module "/pliant/graphic/image/rotate.pli"
...
var Link:ImageRotate r :> new ImageRotate
var ExtendedStatus s := r setup img "rotate_left"

Possible rotations are 'rotate_left' 'rotate_right' and 'rotate_180',
and the underlying image has to be ImagePixmap, nothing else.
The 'bind' alternative is not provided.

Reading the code shows that it's just piece of crap since it uses 'pixel' method instead of 'rectangle_read_map' that make is not only non general, but also poorly optimized.
In very few words: don't use it.

ImageTransform

Here is the more general version of ImageResampling and ImageRotate:

module "/pliant/math/transform.pli"
module "/pliant/graphic/image/transform.pli"
...
var Transform2 m := ...
var ImagePrototype p := ...
Link:ImageTransform t :> new ImageTransform
var ExtendedStatus s := t bind img m p ""

There is no 'setup' alternative.
In the 'bind' prototype,
'm' is the transformation matrix to apply to the underlying image,
'p' is the target image where we plan to draw to the transformed image 't'.

The transformed image 't' will be a subpart (same resolution, but columns and rows removed) of 'p' just big enough to receive all pixels from the underlying image 'img' after applying the transformation.

In the transformation matrix, an angle of pi/2 means the image will be rotated 90° clockwise (because Pliant assumes that the zero coordinates are top left).
Here is an example for it:

var Transform2 m := transform 0 0 1 1 pi/2 pi/2

the two first parameters of 'transform' are the translation, the two next the scale, and the two last the rotation. The scale and rotation are applied before the translation.

When reading the transformed image, some pixels of the transformed image can be mapped to some non existing pixels (outside of the boundaries) in the underlying image. It is safe to read these pixels, and the result will be zero filled pixels.
Now, in some situations where the application does not want to handle pixels that come from nowhere, so handle them as transparent pixels, it is possible to use 'line_clip' before 'read' in order to exclude pixels coming from nowhere.
In other words, 'line_clip' is an indirect way to create an alpha channel:

var Int x y count
...
t line_clip x y count

on return x might be increased, and count might be decreased so that pixels ranging from updated x to updated x+count-1 are granted to be pointing existing pixels in the underlying image.

ImageLut

module "/pliant/graphic/image/lut.pli"
...
var Array:(Array Float32 256) lut
...
var Link:ImageLut l :> new ImageLut
var ExtendedStatus s := lut bind img lut

or

var Curve c
...
var ExtendedStatus s := l bind img c 100 100 ""

or

var ExtendedStatus s := l setup img "exposure 0.02 exposure0 -0.01"

The first version is the most general. For each dimension of the underlying color model, the corresponding lookup table is applied. Each lookup table contains 256 elements and the target value is expected to be a floating point value in the 0 to 255 range.
Please notice that alpha channels will not be modified.

With the second version, the same lut will be computed for each dimension, according to the single provided curve.
The two parameters express the range of values used on the X and Y axis by the curve.
If the 'monotonous' keyword is provided in the parameter string, then the curve will be checked for ascending monotony.
If 'invert' keyword is provided, than the reverse curve will be used.

With the third version, lookup tables will be provided according to a set of instructions.
The most frequently used instruction is 'exposure' that enables to adjust the 50% area without changing the 0% and 100%. With 0.02 value provided, 50% will be transformed to approximatively 52%.
Available instructions are 'middle' 'density' 'exposure' 'multiply' 'add' 'minimum' 'maximum' and 'cut'
Each instruction can be followed by an index (no space between) and it will make it specific to the specified component (the first component is assigned index 0).
'multiply' and 'density' do the same thing.
'cut' receives two parameters. The first one one defines the level bellow which the value will be cut to 0, the second defines the level above which the value will not be changed, and the middle part of the curve will be extrapolated between the two.
See the code for extra details.

Lut code is smart since it will do dithering in order to avoid quantization artifacts, except if 'setup' is used to do the configuration and 'fast' is provided in the set of instructions..
Lut code also assumes (and checks) that the color model is 8 bits per component.

ImageLazy

module "/pliant/graphic/image/lazy.pli"
...
var Link:ImageLazy l :> new ImageLazy
var ExtendedStatus s := l bind "file:/tmp/test.png" ""

or

var Stream s
...
var Link:ImageLut l :> new ImageLut
var ExtendedStatus s := l bind s ""

or

var ExtendedStatus s := l setup (null map ImagePrototype) "file [dq]file:/tmp/test.png[dq]"

An image read filter will be attached to the image, so that reading the lines from the image, as if it was already loaded, will transparently load the lines from the disk.

The string provided as the last parameter can be used to provide parameters to the read filter. See bellow for details about the set of options supported by each read filter.
The 'filter' option can be used to overwrite the filter to select, which is otherwise deduced from the file name:

var ExtendedStatus s := l bind s "filter [dq].png[dq]"

There is one huge constrain about lazy images: pixels must be read in the lines ascending orders.
If this is not satisfied, the program will stop immediately with an error.
Yet, reading a few lines backward is possible provided 'backward' option has been provided at image construction time. If can be useful is the image is used as the underlying image for a filter image like ImageAntiAliasing:

var ExtendedStatus s := l bind "file:/tmp/test.png" "backward 4"

Then, since the 'read' method on an image does not return any status value, after reading one line from the image, testing if reading pixels from the on disk encoded image succeeded will be done through.

if (l query "status")="success"
  ...

ImageRIP

module "/pliant/graphic/image/rip.pli"
...
var Link:DrawPrototype d
...
var Link:ImageRIP r :> new ImageRIP
var ExtendedStatus s := l bind l ""

The 'd' variable is containing a vector drawing. Vector drawing is the second layer of Pliant graphic stack.

Ripping means turning the drawing to an image. The resulting image is generally not built in one time, but rather tile by tile.

If the vector drawing uses a color model with one or more alpha channels, 'drop_transparency' option can be used to discard them on the final image:

var ExtendedStatus s := l bind l "drop_transparency"

Ripping can be a very complex and time consuming process, so there are a few tuning options available that might worth investigate for your specific application:

First, we can rip several tiles in parallel, one per available processor core:

var ExtendedStatus s := l bind l "burst"

Or we can force the number of tiles to be ripped in parallel to let's say 3 through:

var ExtendedStatus s := l bind l "burst 3"

But the best is generaly to use 'balance' option that will adjust automatically according to the load of the Pliant process.

var ExtendedStatus s := l bind l "burst balance"

Then, we can adjust the size of a ripping tile through 'cache' option that specifies the amount of memory to assign to each tile. The default is 4 MB per tile, and performances can often be significantly improved through increasing it, provided the application still gets enough memory available for the Pliant global cache.

var ExtendedStatus s := l bind l "cache "+(string 64*2^20)

The size of each tile can be further more reduced through 'step' option, also I don't remember anymore what it can be useful for:

var ExtendedStatus s := l bind l "step 512"

Finally, there is also a packed option that requests tiles to be in memory packed images. It might be used to render all the image at once. In the following sample, 'cache' option is used to disable the tile size limit so get a single tile in the end, and 'packed_cache_size' is an ImagePacked parameter that will force the tiles of the packed image (your head might start smoking at this point) to swap to a temporary file if the compressed image grows beyond 128 MB. Just don't use 'packed' option if you don't know exactly what to do:

var ExtendedStatus s := l bind l "packed cache ? packed_cache_size "+(string 128*2^20)

About the code, 'do_rip' is the function truly ripping one of the tiles, and 'access' is the function that will start extra ripping threads when burst mode is used.

Write filtering images

ImageClip

ImageClip is intended to provide write masked access to an underlying image.

module "/pliant/graphic/image/clip.pli"
...
Link:ImageClip c :> new ImageClip
var ExtendedStatus s := c bind img 0 0 100 100 "" (var Link:ImagePrototype mask)

The first first four parameters specify a clip box (unit is millimeter). So, when writing to the underlying image through the filter, I mean when issuing 'write' to 'c' filtering image, no pixel of the underlying image will be modified outside the clip box.
Furthermore 'bind' constructs a mask image and returns a link to it in the 'mask' field.

VERY IMPORTANT: The 'mask' image does not behave as an alpha channel.
If the value of a pixel is 'mask' is neither 0 nor 255, the behavior of ImageClip will not paint on the underlying image some densities that are the mixture of the old and new densities; It will rather paint the new densities unmodified, but with alpha channels values adjusted.

Let's take an example.
Assuming that the underlying 'img' image is using cyan + magenta + 2 alpha channels (4 bytes per pixel) color model, and the underlying pixel value is:
   cyan = 0, magenta = 128, alpha1 = 255, alpha2 = 64
and we write to 'c':
   cyan = 128, magenta = 255, alpha1 = 255, alpha2 = 128
with the mask value for that pixel being 128;
then, the result will not be to apply an alpha transparency channel and write in underlying image 'img':
   cyan = 64, magenta = 192, alpha1 = 255, alpha2 = 96
but rather to keep the densities unmodified, adjust the transparency channels, and so write to 'img':
   cyan = 128, magenta = 255, alpha1 = 128, alpha2 = 64

Clipping code assumes (and checks) that the color model is 8 bits per component.

ImageTransparency

module "/pliant/graphic/image/transparency.pli"
...
Link:ImageTransparency t :> new ImageTransparency
var ExtendedStatus s := t setup img ""

The underlying image 'img' must use a color model with no alpha channel, and the 't' image will have multiple alpha channels (one per component in the underlying image).

When writing to 't' image, alpha channels informations will be applied.

Back to 'ImageClip' strange behavior, if you write:

Link:ImageTransparency t :> new ImageTransparency
var ExtendedStatus s := t bind img ""
if s=failure
  ...
Link:ImageClip c :> new ImageClip
var ExtendedStatus s := c bind t 0 0 100 100 "" (var Link:ImagePrototype mask)

then the behavior of 'c' is now the expected one. Now 'mask' behaves as an alpha channel on 'img', but this is handled at 't' level, not at 'c' level

Transparency code assumes (and checks) that the color model is 8 bits per component.

On disk file formats

The module implementing file filters is /pliant/graphic/image/io.pli
The module loading most drivers at once is /pliant/graphic/image/all.pli
There are three ways to read to write an image to disk.

First method:  standard loading or storing

module "/pliant/language/unsafe.pli"
module "/pliant/graphic/image/prototype.pli"
module "/pliant/graphic/image/pixmap.pli"
module "/pliant/graphic/filter/io.pli"
...
var Link:ImagePrototype img :> new ImagePixmap
var ExtendedStatus s := img load "file:/tmp/test.png" ""
if s=failure
  console "loading image failed (" s:message ")" eol

var ExtendedStatus s := img save "file:/tmp/test.jpeg" ""
if s=failure
  console "saving the image failed (" s:message ")" eol

When loading, filter option can be used to force the file filter selection otherwise based on the file name:

var ExtendedStatus s := img load "file:/tmp/test.img" "filter [dq].png[dq]"

When saving, filter option is available, but also the burst and balance options. This will read lines from the image in parallel. This can speed up things significantly on a multicore processor in case the image would be a color conversion or anti-aliasing filter:

var ExtendedStatus s := img save "file:/tmp/test.jpeg" "burst balance"

And finally, when saving, continue_flag option can be used to provide remote control. Basically, the flag will be checked for true before writing each line, so if another thread changes it, writing the image will stop in the middle.

var CBool flag := true
...
var ExtendedStatus s := img save "file:/tmp/test.jpeg" "continue_flag "+(string addressof:flag)

Second method: using a lazy image as described previously in this document

module "/pliant/language/unsafe.pli"
module "/pliant/graphic/image/prototype.pli"
module "/pliant/graphic/image/lazy.pli"
module "/pliant/graphic/filter/io.pli"
...
var Link:ImagePrototype img :> new ImageLazy
var ExtendedStatus s := img bind "file:/tmp/test.png" ""
if s=failure
  console "binding image failed (" s:message ")" eoinding image failed (" s:message ")" eol
  return
var ExtendedStatus s := img save "file:/tmp/test.jpeg" ""
if s=failure
  console "saving the image failed (" s:message ")" eol

Third method: Low level interface

module "/pliant/language/unsafe.pli"
module "/pliant/graphic/image/prototype.pli"
module "/pliant/graphic/filter/prototype.pli"
...
var Link:ImageReadFilter f :> image_read_filter ".png"
if not exists:f
  console "PNG images filter is not available" eol
  return
var Link:Stream stream :> new Stream
stream open "file:/tmp/test.png" in+safe
if stream=failure
  console "failed to open image file" eol
  return
var ExtendedStatus s := f open stream "" (var ImagePrototype h)
if s=failure
  console "opening the image failed (" s:message ")" eol
  return
var Address b := memory_allocate h:line_size null
part read_lines
  for (var Int i) 0 h:size_y-1
    if (f readline b)=failure
      console "failed to read line " i+1 "/" h:size_y eol
      leave read_lines
    ...
memor_free b
if f:close=failure
  console "failed to terminate reading image" eol

var Link:ImageWriteFilter f :> image_write_filter ".png"
if not exists:f
  console "PNG images filter is not available" eol
  return
var Link:Stream stream :> new Stream
stream open "file:/tmp/test.jpeg" out+safe
if stream=failure
  console "failed to open image file" eol
  return
var ImageProtype h
...
var ExtendedStatus s := f open stream "" h
if s=failure
  console "opening the image failed (" s:message ")" eol
  return
var Address b := memory_allocate h:line_size null
part write_lines
  for (var Int i) 0 h:size_y-1
    ...
    if (f writeline b)=failure
      console "failed to write line " i+1 "/" h:line_size eol
      leave read_lines
memor_free b
if f:close=failure or stream:close=failure
  console "failed to terminate storing the image" eol

Please notice that in the low level method, the stream must be an object, not just a local variable.

Fouth method: fast loading and saving pack4 encoded images

See in memory compressed images paragraph earlier in this document.

Here is now the description of special properties of each image reading and writing filter:

ppm and pgm file format

The simplest well know file format for encoding an RGB image.
Does not support compression, nor specifying image resolution.

As a result, when reading the resolution will have to be provided, or 72 dpi will be assumed:

var ExtendedStatus s := img load "file:/tmp/test.ppm" "resolution 300"

or, if the horizontal resolution is 300 dpi but the vertical resolution is 600 dpi:

var ExtendedStatus s := img load "file:/tmp/test.ppm" "resolution 300 600"

A 'negative' option is also supported by the read an write filters, than reverse (apply 255-x transformation) each byte value:

var ExtendedStatus s := img load "file:/tmp/test.ppm" "negative"

and:

var ExtendedStatus s := img save "file:/tmp/test.ppm" "negative"

png file format

PNG is the recommended file format for lossless images.

Only resolution option is supported when reading (see example in ppm and pgm filter), and it will be used only when the resolution is not provided in the png file (resolution is an optional tag in PNG file format).

jpeg file format

This is not native Pliant implementation but rather in interface to libjpeg.

The very important option when writing a JPEG image is the quality. A quality of 1 would mean perfect image, but makes since the format is not optimized for that, and a quality of 0.25 writes an ugly image. Make your own tests (I personally consider 0.5 as low quality, and 0.9 as high quality).

var ExtendedStatus s := img save "file:/tmp/test.jpeg" "quality 0.9"

Resolution option is supported when reading (see example in ppm and pgm filter), and it will be used only when the resolution is not provided in the png file (resolution is an optional tag in JPEG file format).

Then, JPEG format supports comments, so you can add a comment at write tume through:

var ExtendedStatus s := img save "file:/tmp/test.jpeg" "comment [dq]Hello[dq]"

Reading the comment is ... not possible. Who is the stupid writer of this filter ?
If you need to read or write comments to JPEG images, an out of the image libary /pliant/appli/photo/jpeg.pli module is provided with 'jpeg_set_comment' and 'jpeg_get_comment' functions.

tiff file format

TIFF file format is mostly used by professional softwares for encoding CMYK images.
This is not native Pliant implementation but rather an interface to libtiff.

The main TIFF option is the one specifying the compression to use. Supported value are 'packbits' 'lzw' and 'zlib'. 'packbits' is the faster one, 'zlib' the more efficient, but 'zlib' is an extension not supported by old readers. I have not mapped other compression methods such as the various CCITT since they have been designed for monochrome (1 bit per pixel) images and Pliant image library is not designed to handle this kind of images.

var ExtendedStatus s := img save "file:/tmp/test.tif" "packbits"

Resolution option is supported when reading (see example in ppm and pgm filter), and it will be used only when the resolution is not provided in the TIFF file (resolution is an optional tag in TIFF file format).

Maybe I should add code to automatically turn 1 bit per pixel images to 8 bits per pixel when reading, and enable writing 1 bit per pixel images either through transparent thresholding or dithering.

packed Pliant file format

This is the preferred file format for storing very high resolution (more than 600 dpi) text+image documents.

Supported options are 'plan' that will store the image as a set of plans instead of as a set of pixels, and it will improve compression in some situations, at the expense of speed, and 'tile_x' and 'tile_y' that force writing tiles instead of lines (don't use tiling options unless you know what you do):

var ExtendedStatus s := img save "file:/tmp/test.packed" "plan tile_x 512 tile_y 128"

Please notice that the on disk packed file format is currently an ASCII (UTF8 in facts) header, followed by an empty line, and then binary datas, and it should be changed to be PML encoded.

As a summary of all supported file formats, if you want to store RGB images on disk, you should either select PNG or JPEG depending if you expect lossless or lossy compression, unless the resolution is very high.

Console drivers

We can see the UI client as a five layer software:

So, the console drivers could be presented as another layer in a different document, but since it mostly deals with images and I don't plan to provide a lot of details since it's only used as the hardware glue code for Pliant UI client, I decided to include it here.

Console drivers prototype is defined in /pliant/graphic/console/prototype.pli

The only two functions for setting the screen content are:

method c paint img tx ty
  oarg_rw ConsolePrototype c ; oarg_rw ImagePrototype img ; arg Int tx ty

and

method c copy x0 y0 x1 y1 xx yy
  oarg_rw ConsolePrototype c ; arg Int x0 y0 x1 y1 xx yy

It means that the UI is really using the Pliant graphic stack to render the content on any platform, then call 'paint' to transfer the result on the screen, so that the viewed document will be pixel to pixel the same on all platforms.
The only hardware accelerated feature used by the UI is scrolling, and it is implemented through using the 'copy' function. Using hardware acceleration for implementing smooth scrolling is still required, even on multi Ghz modern processors, because the access to video memory is too slow.
In very few words, a modern processor can do software only very decent 2D rendering for document editing applications, with two exceptions: full anti-aliasing that would need to hardware accelerate the all vector drawing instructions set (not provided by Pliant graphic stack at the moment), and scrolling that requires hardware copy.

Then, there is a single function for receiving mouse and keyboard events:

method c event key buttons x_or_x0 y_or_y0 x1 y1 options -> event
  oarg_rw ConsolePrototype c ; arg_w Str key ; arg_w Int buttons x_or_x0 y_or_y0 x1 y1 ; arg_w Str options ; arg Str event

on return, event value will be 'character' or 'uncharacter' or 'press' or 'release' or 'move' (or something else and rare that I forgot to document).
A mouse click will be reported as event="press" and key="button1"
'options' field will receive non standard informations such as the hardware position of a pressed key.

Console drivers currently supported are:

The HTTP proxy is not a console driver since it does not provide the methods described in this paragraph, but it also plays the role of a possible UI fontend.

VNC RFB protocol is working very nicely as a Pliant console driver because (as opposed to Linux framebuffer) they very well designed the instructions set, which is ... the same as Pliant console with only 'paint' and 'copy' but with several compression mechanisms to help reduce the traffic between the client and the server (1).

The Linux kernel framebuffer frontend works poorly (unless and even using a patched tree) because Linux kernel framebuffer is very poorly maintained (2). No high end Linux guy is using it since they all use X11, so that the framebuffer is completely changing every few months (for better consistency as they say), but each of the rewrites stops as soon as X11 works nicely on top of it and never end as a properly working framebuffer.
They also don't seem to understand that copyarea function cannot be implemented at software level and insist in not exposing the hardware acceleration function to applications. Please notice that X11 has exactly the same constrains as Pliant UI: you could have a perfectly working (responsive) system working with X11 on top of Linux framebuffer ... provided the single hardware accelerated copyarea function is exposed by Linux framebuffer and used by X11.
If you are a good advocate, please help.

The conclusion is: there is not much freedom when designing a generic console layer, so all people that spend some time experimenting end to basically the same. Please Linux framebuffer ... join us and finish you part of the work, which is hardware drivers.

I don't plan to provide great details about each console driver implementation since most of them are fairly simple glue code and they are unlikely to be used as anything else than Pliant UI client front-end.

Printer drivers

Theory

Let's start by what a printer interface SHOULD be.

A printer should be defined by:

Then sending a document to the printer should be sending a PACK4 encoded bitmap with the right number of 8 bits components and at one of the supported resolution. No more.

The job of the printer would be to do anti-aliasing, splitting inks to cartridges (as an example when there is both a cyan and light cyan cartridge) and dithering (3).

This model works great for both low end printers and very high end one that print many pages per minute.

But for a high end printer with a RISC processor, Postscript is better since it frees the PC from calculations ?

No. For three reasons:

So, while introducing the various printer drivers supported by Pliant, I will comment on how far they are from the optimal model just described.

Application interface

At application level, driving a printer is the same as writing files on disk. So the four methods described in 'On disk file formats' paragraph apply also to printers.
Also, the image resolution and color model MUST match printer specifications.
Here are a fiew samples:

var ExtendedStatus s := img save "device:/usb/lp0" "filter [dq].escp2[dq] model [dq]Epson R800[dq]"

or:

module "/pliant/protocol/lpr/client.pli"
...
var ExtendedStatus s := img save "lprng://192.168.0.30/lp" "filter [dq].pcl[dq] pcl6"

'device:/usb/lp0' does not work under Windows because under Windows an USB connected printer is not accessible directly, as opposed to a parallel port connected printer.
Ghostscript provides a trick to work around, Pliant currently does not; so you're left in the cold at the moment.

LPR protocol specifications published in RFC1179, 18 years ago, specifies that the size of the data file is provided at the beginning of the content. This is not convenient since it prevents to send the file on the fly (because in such a situation the size of the file is not known soon enough). As a result, RFC1179 states that zero shall be provided and it means that data file end will be TCP connection termination.
Anyway, HP print servers tend to still not implement this correctly. Hey, wake up HP, you are number one, aren't you ? Well, since they might not ear, just replace 'lprng' with 'lpr' and Pliant will transparently store the file locally before sending.

Let's now list and comment various provided printer drivers:

escp2

This driver supports most Epson inkjet printers.

The problem with ESCP2 language is that it's too low level. You have to provide one or two bits per cartridge per pixel. It means that dithering has to be performed on the computer side, and it results in a too big file.

The other problem is that, also Epson inkjet printers technologies mostly stopped significantly evolving years ago (basically, with the 3 pl ink pigmented printers generation, I mean Stylus photo 2000, 4000, 7600, 9600), they still tend to change some details with each model (beyond changing nozels number and spacing and extending cartridge selection instruction) that makes upgrading the driver mandatory ... for nothing (except maybe have the RIP sellers make more money through upgrades).
They still also don't know how to select the right set of inks, so change it mostly randomly with each model, but that's another story.

They also seem to have conflict between handling features (like the cutter) at ESCP2 instructions set level, or at the no use job-ticket EJL layer they added on top of it at some point.

Back to concrete, when using the Pliant ESCP2 driver, you have to provide 'model' option is in the example above, but it is likely that your model be not supported, so you end through selecting a not too different one, than provide extra options to force some other settings.
They are so many of them, that I will not try to provide the complete list in this document. Read the code or send me a mail.

Anyway, here is a subset of the options you can use with this driver:
First, if you get poorly aligned drawing on the page, you might want to use 'offset' option (the horizontal then vertical offset is specified in millimeters):

var ExtendedStatus s := img save "device:/usb/lp0" "filter [dq].escp2[dq] model [dq]Epson R800[dq] offset 5 5"

Then ... there is no then at the moment.

About Epson driver implementation, if you read the code, you will see that the complex part is dithering because having two cartridges with the same color (cyan and light cyan) and three possible dot sizes on each cartridge makes dithering much more complex than using a single cartridge and a single dot size.

The general idea of Pliant one bit dithering (as opposed to the one used for dealing with quantification issues and used by several image filters such as ImageLut introduced earlier in this document) is just to use a very large thresholding matrix.
It avoids the dirty artifacts of remaining propagation algorithms such as the well known Floyd and Steinberg through properly spacing dots when there are very few of them (density close to zero), and also avoids the artifacts of regular thresholding matrix as used on early Windows systems.
The nice trick of it is that it consumes a lot of computing power, but consumes it once for all when building the matrix. It's implemented in /pliant/graphic/misc/dither.pli

Back to dithering implementation in ESCP2 driver, among the 6 dot levels available (for the cyan as an example, we have 3 of them using the dark cyan cartridge, and 3 of them using the light cyan) we use only 3 of them. The general reason for using few is that the more levels you use, the more likely you are to fall on not smooth enough transition issues. The three levels we use are a big dot on the dark cartridge, or the smallest one on the dark cartridge, or the smallest one on the light cartridge. Deciding among the 3 dot levels is performed in the code block marked with the comment 'handle pixel x y on head h , which level is l' in the commented out code block marked as 'unoptimized version'.

Setting ESCP2 driver ink adjustment parameters escp2_density, escp2_middle and others is intended to provide as much ink as possible at 100%, and a flat density curve (the density curve is drawn from computing the effective density from the output of the spectrocolorimeter), within the limit of having not too much ink at 100% on all colors. It can be tricky if the relative size of various dot levels and pigments proportion between the light and dark cartridge are not the expected ones as a result of a model change. In such a case, it might require adjusting transition parameters such as escp2_light_removal_start, escp2_light_removal_power and others, so ... it's outside the scope of this initial documentation. Too much is too much.

The other operation that the driver has to perform on a desktop Epson inkjet printer (as opposed to professional models that do it on printer side) is what they call weaving.
The general idea is that on an Inkjet printer, you never print all the dots from the first tile, then move the head forward and all dots from the second one, and so on. The reason is that it would result in awful banding as a result of not enough head positioning accuracy. The solution is to print only some of the dots of each line, and have ink nozels spaced by more than one line so that you also don't print all the consecutive lines on one pass, then do several pass. The paper is moving up a regular and small distance between each pass of the head.
On low desktop Inkjet Epson printers (up to models 2xxx), the printer does not have enough memory to store all the dots value, so the driver has to do weaving on the PC side and provide just the dots that will print on the current pass. On the other hand, on professional models (starting from models 4xxx), the printer has enough memory, so the driver sends lines one after the other, and the printer deals with spreading dots to multiple pass according to the strategy (a big word for not much in fact) requested through 'escp2_microweave' parameter.
At implementation lever, 'space_x' and 'space_y' are computed from effective requested resolution, printer head frequency and nozels spacing, to match the number of dots between to printed ones the same pass, horizontally and vertically. If the printer is dealing with weaving itself, we will set 'space_x' and 'space_y' to 1 that will result in just disabling weaving at driver level through providing all dots of a tile at once.

Another important setting to think about is 'unidirectional' option. If you set it, the head will print only from left to right, so printing will be slower because the time to move head back to the left will be lost, but head positioning will be more precise, so the final result will be sharper.

pcl

Provides PCL5 and PCL6 support.
This driver can be used to drive most laser printers.

This is the preferred language for most laser printers.
PLC6 is the same as PLC5 with instructions being binary encoded as opposed to ASCII, and it makes decoding faster on printers short of computing power.

PLC is a vector drawing language, but Pliant driver will send an image instead. See 'theory' subparagraph for justification.

Just like ESCP2, PCL has a PJL jobticket language on top of it, that just like with ESCP2 worth nothing but troubles because most printer will want something special at PJL level or just not work ... even if PCL5 or PCL6 compatibility is claimed on the advertising papers.

The good news about PLC5 is that it provides a compression mechanism named 'Seedrow' that is a very decent one (packbits like working on pixels instead of bytes, and with a same as on the previous line extension, that makes it not too far from Pliant PACK4 capabilities).

Now the bad news are:

About implementation, the code is a bit simpler than with ESCP2 because no complex dithering is required. Anyway, providing 'pcl_dither' option switches the driver to use 1 bit per component instead of 8, so use Pliant matrix thresholding based dithering.

ijs

IJS is a standard interface introduced by HP to provide the document to print as an image to a server (an executable program in facts) that is providing drivers for many printers.

IJS interface is very much like Pliant one, so Pliant glue driver is fairly short.

IJS is expecting at least one 'model' option to decide the IJS driver to use:

var ExtendedStatus s := img save "device:/usb/lp0" "filter [dq].ijs[dq] model [dq]hp color LaserJet[dq]"

See IJS documentation for the list of models available in your IJS server.

If the IJS software to use is not 'hpijs', it can be changed using 'ijs_server' option.
If the printer manufacturer to provide is not 'HP', it can be changed using 'manufacturer' option.
If the image provided to the printer is not encoded in the printer specific RGB (I assume printer specific RGB means printer CMYK transformed to RGB using entry level rules such as R=1-C), then you should use 'sRGB' option.
Other options are 'ijs_quality' defaulting to 2, 'ijs_colormode' also defaulting to 2, 'ijs_mediatype' defaulting to 0, 'ijs_penset' defaulting to 2 and 'ijs_papersize' defaulting to '8.27x11.69'.
For extra details about these parameters, the code should be easy to read.

Roughly speaking, IJS is designed for desktop printing on (mostly HP) low end inkjet printers that have so few computing power that they need the PC to do dithering (as opposed to printers using PCL5 or PCL6).

gimpprint

Gimpprint is the first free library that tried to provide high quality printing on inkjet printers.

I have to thank Gimpprint guy Robert L Krawitz for documenting Epson printers at a time Epson was not providing detailed specifications.

Anyway, Gimpprint is using too smart dithering algorithms that end in just disturbing color management, and still very primitive color calibration techniques.

Gimpprint also use a 'model' option.
It accepts a 'page' option with to values providing horizontal and vertical dimensions in millimeters, the default being 210 by 297.
It also accepts an 'offset' option with two parameters to translate the content on the page.

The Pliant gimpprint glue code should not be too hard to understand, except that they use plenty of callback functions instead of enabling the external software to provide them lines one after the other.

You might want to try Gimpprint driver if you need to drive a supported Canon inkjet printer.

Conclusion

If you want desktop printing, a PCL compatible printer driven using Pliant native PCL driver is probably the way to go, and if you want high fidelity photos, you need an Epson printer with pigmented inks, and a color profile for it (with the right paper), but building a color profile requires a spectrocolorimeter so is not what average Jo user can do.