Building Data-Driven Websites Using the Wolfram Language

Dr. Oliver Grasl, transentis consulting

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

ABSTRACT

Learn how to build interactive, data-driven websites using only the Wolfram Language.

This is simple thanks to the HTMX framework, which let’s you build modern web-based UIs utilizing the power of hypertext.

Hypertext can easily be generated using Wolfram Language’s Symbolic XML and API-Functions.

Harnessing the Power of Wolfram Language for the Web

The Wolfram ecosystem already has powerful features that support web development:

Symbolic XML to create web content

API-Functions to make functionality accessible on the web

Wolfram Cloud to deploy APIs and Content to the cloud

Wolfram Web Engine to deploy flexible web servers

Wouldn't it be great if we could use Wolfram Language to create powerful, interactive websites!

The Challenge

Wolfram Language doesn't run in the browser

The key challenge to building highly-interactive, data-driven websites is that the Wolfram Language doesn't run natively in the browser – in fact currently browsers only support HTML, CSS and Javascript natively .

Key Idea: Generate Content on the Server

The only way to deal with this is to generate content on the server.

But modern Web Apps are quite complex - we want to be able to update “parts” of a page without having to re-render the whole page every time.

How can this be achieved?

HTMX To The Rescue

The HTMX framework let’s you build modern web-based UIs with the simplicity and power of hypertext.

URL["https://htmx.org"]

It gives you access to AJAX, CSS Transitions, WebSockets and Server Sent Events directly in HTML using HTML attributes .

It is a small library that can be directly embedded into webpages .

It is dependency-free and doesn't need a compile step .

Simple Examples Combining WL and HTMX

Let’s build the example above – taken from the HTMX website –, using Wolfram Language.

We start with the overall page structure outline:

simplePage:=XMLObject["Document"][
{XMLObject["Doctype"]["HTML"]},
XMLElement[
"html",{},
{
head,
body
}](*html*),
{}
];

In the page-head, we load the HTMX library. The library is small and can be downloaded directly.

head:=XMLElement["head",{},{XMLElement[
"script",
{
"src"->"https://unpkg.com/htmx.org@1.9.11","integrity"->"sha384-0gxUXCCR8yv9FM2b+U3FDbsKthCI66oH5IA9fHppQq9DDMHuMauqq1ZHBpJxQ0J0","crossorigin"->"anonymous"
},
{" "}
](*script*)
}];

The page-body contains the interactive code: in this case a button, which calls an API-Function in the WolframCloud when clicked (we are using the WolframCloud as a webserver here).

Note the use of the HTML hx-post and hx-swap attributes.

body:=XMLElement["body",{},
{
XMLElement[
"button",{
"hx-post"->"https://www.wolframcloud.com/obj/transentis/wtc2024/demo/content",
"hx-swap"->"outerHTML"
},
{
"click me"
}](*button*)
}];

The API-Function just returns some hypertext:

cloudContent := CloudDeploy[APIFunction[
{},
"<h1>This is the content.</h1>"&,
"HTML"
],
"wtc2024/demo/content",
Permissions->"Public"
];

The text sent back from the Wolfram Cloud is “patched in” to the web page by the HTMX framework, replacing the button.

Let' s try this:

EmbeddedHTML[ExportString[simplePage, "XML"],"CompletePage"->True]

Voila - our first interactive webpage, built using HTMX and Wolfram Language!

A Slightly More Sophisticated Example

Let’s build a web-page for this interactive plot:

Manipulate[Plot[Sin[x*(1+a *x)],{x,0,6}],{a,0,2}]

We’ll use an API Function to render the plot as an SVG and pass that back to the webpage:

cloudContent := CloudDeploy[APIFunction[
{"a"->"Real"},
ExportString[Plot[Sin[x(1+#a* x)],{x,0,6}],"SVG"]&,
"HTML"
],
"wtc2024/demo/sin",
Permissions->"Public"
];

Let' s test the function:

URLExecute["https://www.wolframcloud.com/obj/transentis/wtc2024/demo/sin?a=1.0"]

Now we need a page that will display the plot, provide a slider to set the parameter and a run button to update the page.

The page itself has the same structure as before:

plotPage:=XMLObject["Document"][
{XMLObject["Doctype"]["HTML"]},
XMLElement[
"html",{},
{
head,
plotBody
}],
{}
];

The body contains a HTML div-Element that contains the dynamically updated plot - we preload an initial plot with setting a=1.

The body also contains a HTML form-Element that provides the controls.

plotBody:=XMLElement["body",{},
{
XMLElement[
"div",
{
"id"->"thePlot",
"hx-get"->"https://www.wolframcloud.com/obj/transentis/wtc2024/demo/sin?a=1",
"hx-trigger"->"load",
"hx-swap"->"innerHTML"
},
{
XMLElement["p",{},{"Loading initial plot..."}]
}
](*div*),
XMLElement["form",
{
"hx-get"->"https://www.wolframcloud.com/obj/transentis/wtc2024/demo/sin",
"hx-params"->"a",
"hx-swap"->"innerHTML",
"hx-target"->"#thePlot"
},
{
XMLElement[
"input",{
"type"->"range",
"min"->"0",
"max"->"2",
"value"->"1",
"step"->"0.1",
"class"->"slider",
"name"->"a"
},
{}
](*input*),
XMLElement[
"input",
{
"type"->"submit",
"value"->"Run"
},
{
}](*button*)
}](*form*)
}];

EmbeddedHTML[ExportString[plotPage, "XML"],"CompletePage"->True]

How to Build A Data-Driven Website Using Only WL and HTMX

Key Wolfram Language features used

Symbolic XML to create web content

API-Functions to make functionality accessible on the web

Key Wolfram Technologies

Wolfram Web Engine to provide a web server that wraps the application build using WL

Wolfram Engine to host the application

3rd Party Technologies

HTMX for partial updating of webpages via AJAX (asynchronous Javascript and XML

Tailwind CSS (to style the webpages)

Optional: Observable Plot (for Interactive Line Charts)

The Initial Wolfram Notebook And the Resulting Website

The application we are building is a frontend for an enterprise digital twin, that provides a real-time view of KPIs now and simulates how they might develop in the future.

Initially we built the UI using a Wolfram Notebook

See the separate Wolfram Notebook.

The resulting website

You can access the website we built here: twin.transentis.com

Read more about the backend simulation here : bptk.transentis.com

Overall Architecture

Frontend built using WL, served via Wolfram Web Engine

Simulation Engine runs as a separate service, provided by a Python Backend

Anatomy of A Typical Webpage

Break pages down into components.

Each component is served by a hypermedia call, i.e. a parameterized call to a URL / API-Function  that renders a HTML fragment.

Relevant parameters are passed back and forth using disaggregated associations.

Encode state into URL query parameters as far as possible.

Store complex state on the server via Cookies.

If necessary, use Once-Functions to ensure code is only initialized once.

Challenges A Long The Way

Handling Session State

The key challenge was how to handle session state - e.g. to remember settings when switching between tabs.

We used session cookies for this and built our own cache data structure to attach the session state to the cookies in the backend.

XML vs XHTML vs HTML

Symbolic XML generates XHTML, not HTML – we ran into some subtle points here, especially regarding CDATA Sections.

Q & A