High interactivity applications using Pliant UI

High interactivity features are used to provide instant reaction to user input in database applications (hook change) and to implement more interactive applications such as a text editor or VNC viewer.

Hooks

Hook instruction is used to bring back extra informations from the UI client to the server.
We will see at the end of the paragraph that it can also be used in advanced applications to provide lazy drawing.

The simplest form, and most frequently used variant in database applications, is to request some code to be executed each time some input field is changed by the operator. Here is an example:

ovar Int v1 := undefined ; ovar Int v2 := undefined
hook
  input "" v1 help "Please enter the first number here"
  input " + " v2 help "Please enter the second number here"
  section "result" dynamic
    if v1<>undefined and v2<>undefined
      text " = "
      bold 
        text (string v1+v2)
change
  section_replay "result"

Another example is available in the sample shopping database studied in 'A gentil introduction to using Pliant databases' where we use it to check the customer ID of the order through displaying the customer name.

Then comes the very powerfull, and low level 'hook ... event'.
Should you want real samples, the study /pliant/appli/editor.ui and maybe /pliant/appli/file.ui and /pliant/appli/document.ui

hook
  text "Some content"
event
  section_overwrite "help"
    text "got an event !" ; eol
    text "event = '"+event+"'" ; eol
    text "key = '"+key+"'" ; eol
    text "options = '"+event_options+"'" ; eol
    text "buttons = "+string:buttons ; eol
    text "pointer_section = '"+pointer_section+"'" ; eol
    text "pointer_index = "+string:pointer_index+" ; eol
    text "pointer position is x = "+string:pointer_x+" and y = "+string:pointer_y
    text "focus_section = '"+focus_section+"'" ; eol
    text "focus_index = "+string:focus_index
over
  help "over is "+key
move
  help "position is "+string:pointer_x+" "+string:pointer_y

Please notice that a single hook can have several feedback execution blocs. Here, we have 'event', 'over' and 'move'.

'event' bloc catches mouse and keyboard actions.
- explain the value of various variables usable in event bloc -

'over' catches mouse entering or leaving the hooked area. 'key' variable value will be respectively 'on' and 'off'.

'move' reports mouse movements over the hooked area.

'hook ... click' is a simpler alternative to 'hook ... event' that handles only mouse left button clic, so:

hook
  text "Some content"
click
  ...

is the same as:

hook
  text "Some content"
event
  if event="press" and key="button1"
    ...

The Pliant UI client assigns some special keys sequences to windows management. As an example, Ctrl + F2 is used to switch to the second session. If 'blackhole' optional keyword is used, then the special keys sequences will not be preempted by the windows manager if and only if the mouse pointer is over the hooked area:

hook blackhole
  ...

As an example, 'blackhole' his is used by the VNC client in /pliant/protocol/vnc/index.ui

Now, I'm going to explain the drawing related hooks features:

'visible' enables lazy rendering:

hook
  ...
visible
  ...

'visible' bloc will be executed each time the visible subarea of the hooked area changes. 'visible_x0' 'visible_y0' 'visible_x1' 'visible_y1' variables can be used by the application code to know what area is now visible. This mechanism is named lazy rendering because it enables the server to send (or compute) not all the content of the document at once, but rather send elements only when they get visible. Let's imagine that you want to render a document server side because it uses complex 3D drawing instructions that are not available as part of the UI protocol. Then using this mechanism, the UI server can start by sending an empty document, then be queried for the newly visible parts each time the client scrolls.

Lastly, 'hook' instruction can be used to force server side rendering when the client is a standard web server as opposed to being a native UI client.

hook ssr png noaa
   ...

'ssr' means server side rendering. Server side rendering can be useful in cases where final rendering is not satisfying when the client is a standard web browser because of the limits of the UI protocol to UI to HTML translation. As an example, HTML does not support server provided fonts, and many UI styling features translate approximatively or not at all.

'png' is an optional extra keyword that forces the image to be sent to the web browser as a PNG (lossless) image as opposed to sending it as a JPEG (lossy) image. JPEG tend to be much lighter than PNG, but it's might not if the image is a drawing as opposed to being a real image, and JPEG as dirty artifacts when there is a sharp limit between two evenly colored areas.

'noaa' is another optional keyword that tels the server not to do oversampling when drawing the image. Oversampling (also known as anti-aliasing) generally brings better quality, but it's more computing intensive.

Focus

section "foo"
  para cursor edit
    text "some text"
...
focus_set "foo" 2

'cursor' optional keyword added to 'para' instruction specifies that the paragraph shall receive the focus if the operator click somewhere in the paragraph area.
'edit' optional keyword specifies that if the paragraph has the focus and the operator starts typing, the characters shall be temporary inserted and displayed by the client before the instructions be received by the server. This feature is used to improve operator comfort when the latency between the client and the server is high.

Then, we have 'focus_reset' that just removes the focus. Not very usefull:

focus_reset

'focus_save' and 'focus_restore' are more interesting. They are intended to save the focus before opening a pop up window that will automatically receive the focus, then restore it when the window closes. The objective is to enable the operator to work using only the keyboard in most situations in order to improve everyday users productivity:

button "do something" key "alt d"
  focus_save
  window top
    input "Some field: " (var Str value) focus true
    button "confirm" key "alt c"
      ...
      window top
        void
      focus_restore
    button "cancel" key "escape"
      window top
        void
      focus_restore

Clipboard

An application can write any content in the clipboard using 'clipboard_write', or read the clipboard content using 'clipboard_read'.

Here are two samples that show how to put some text in the clipboard, and how to scan for text in the clipboard:

clipboard_write s
  s otag "text" cb

clipboard_read s
  while s:imore
    if (s itag "text" (var Str t))
      help "Clipboard text content is '"+t+"'"
      s inext
    else
      s iskip

If these listings look cryptical to you, then you should read the article about PML encoding.

Please notice that 'clipboard_read' works asynchronously. A clipoard content will be sent the the UI client, but the body of the instruction will be executed when, and if, the content is received back from the UI client, with the instructions following 'clipboard_read' being executed in the mean time. In other words, 'clipboard_read' works a little bit like a 'button': it records some lazy code that will be executed when and if the content is received just like a button body is executed when and if the button is clicked on the UI client.

The clipboard might be convenient for exchanging some text between two applications, but when exchanging files between two files browsers, it might not be convenient if the files content is in the gigabyte range.
The Pliant lockers have been introduced to provide indirect content. Instead of sending the content of the clipboard to the UI client, the application sets a locker, then send the keys (the name of the machine and the locker password) as the clipboard content. When the target application reads the clipboard, it will receive the keys instead of the content, then will connect directly to the machine containing the locker to get the content.

'locker_create' will fill the 'ticket' (second) argument with a secret key that is needed to access the locker content. The first argument must be an identifier, 's' in the following sample, and it will be the stream the server will use to provide the locker content whenever requested. In other words, 'locker_create' contains a bloc that will be executed lazily, I mean when the locker content is requested, and the bloc execution is responsible to send the content of the locker over the stream.

'locker_use' enables to read the content of the locker from the stream. The second and third arguments specify the computer that contains the locker and the key of the locker. The bloc is executed, and during execution, the stream used to receive the content of the locker is specified by the identifier provided as the first argument.

module "/pliant/language/context.pli"
module "/pliant/util/pml/locker.pli"

locker_create s (var Str ticket)
  s otag "my_direct_content" (repeat 10000 " ")
clipboard_write s
  s otag "my_indirect_content"
  s oattr "server" computer_fullname
  s oattr "ticket" tocket

clipboard_read s
  while s:imore
    if (s itag "my_indirect_content")
      s iattr "server" (var Str server)
      s iattr "ticket" (var Str ticket)
      s inext
      locker_use s2 server ticket
        if (s2 itag "my_direct_content" (var Str t))
          ...
          s2 inext
        else
          ...
    else
      s iskip

Please notice that lockers add a level of complexity, but if the source server is distant from the UI client and so is the target server, the clipboard content stored in the locker will go straight from the source server to the target server. This can be a key advantage for servers remote management.

Some higher level functions have been added on top of 'clipboard_write' and 'locker_create' and on top of 'clipboard_read' and 'locker_use' to enable easy single file upload and download:

clipboard_file_upload s "test.txt"
  s writeline "a sample file"
  s writeline "with two lines"
clipboard_trigger "test.txt" "text/plain"

clipboard_file_download s name
  section_overwrite "report"
    text "file name is: "+name ; eol
    text "file content is:" ; eol
    while not s:atend
      text s:readline ; eol

The first argument of 'clipboard_file_upload' is an identifier that will be the stream you use to write the file content in the clipboard. The second argument is the name of the file in the clipboard.

Then 'clipboard_trigger' is used to force download on the client when it is a standard web browser. The first argument is the name your browser will use when storing the file content on disk, and the second is the mime type. Use 'binary/*' as the mime time if you want to force the browser to download to disk instead of trying to open and display the document content.
On the opposite, when the client is the Pliant UI, the user will use the Pliant UI files browser to decide where he wants to copy the file.

'clipboard_file_download' enables you to read the content of the file in the clipboard. While the body is executing, the first argument is the identifier you can use to read the file content (it will have 'Stream' data type), and the second argument is the identifier you can use to receive the name of the file in the clipboard (it will have 'Str' data type).

The application code for enabling the user to upload a file will generally look like this:

button "process uploaded file" upload
  clipboard_file_download s name
    ...

Please notice the 'upload' option in the 'button' instruction: it instructs the Pliant HTTP proxy to display a file upload input field before the button so that the user can specify the file he wants to upload. On the other hand, when using the Pliant native UI client, the user just selects the file he wants to upload using the Pliant files browser, then press the button to instruct the server to read the clipboard content, download the file and process it.

Last but not least, the Pliant UI clipboard has a very serious security design related issue: any application can read it's content at any time. It will have to be restricted, but I have not found the way to do it without seriously disturbing users.

Maybe both in facts.
In other words, the design problem is that clipboard operation is currently application (server side) driven, whereas it should be windows manager (client side) driven.