Appearance
Configuration
Almost all configuration is done inside of Aeppic documents itself (Which services to run etc). Some settings though that are web server or node specific, are contained in a configuration file. These pertain mostly towards which port to listen to, which SSL certificate to use if any, upload bandwidth limits, overload protections, info as to the returned index.html etc.
The environment the server starts with, is also influenced by systemd usually located at /etc/systemd/system/aeppic-server.service
Security Sensitive
Some settings can have serious consequences when not correctly configured !
Do not change settings like Custom Headers, the Trusted Proxy, or others without understanding the implications.
Location
Default location is /etc/opt/aeppic/server.json
in JSON format.
Fallback defaults can be found in the server.json
located in the main server directory.
HTTP/S Server
These keys configure the HTTP and HTTPS server parts.
The hostname
is used to know which fqdn the server is accessed by. In case of using a self-signed certificate its also the common name. When port
is not null it will activate the http server on that port and on the specified ip
. If hostname is not defined the machine hostname is used.
The https
settings in the https
section are identical, but https needs to be enabled with https.enabled
set to true
. key
and cert
have to point to resolvable pem files. If the paths don't exist a self-signed certificate is used. If the files are not readable the server will raise an exception.
If the server is to be dually hosted under http and https without just redirecting the http traffic to https, the key allowHttpNoRedirect
can be set to true
. This scenario can be helpful if the server is in an intranet and uses https while also being visible from behind a terminating proxy. Or if https is used from the server directly, but the API users such as servcies can work without SSL.
Running aeppic-server just with http is only sensible for some dev scenarios on localhost usually as a lot of web features require https or localhost.
"hostname": "",
"port": 80,
"ip": "0.0.0.0",
"https": {
"enabled": false,
"ip": null,
"port": 443,
"key": null,
"cert": null,
"allowHttpNoRedirect": false,
},
Api Url
In certain scenarios, such as intranet settings, it is helpful to have a way to define which API Url for internal services to use instead of using the public hostname.
"api": {
"url": "http://localhost/api"
}
Data Location
"dataDir": "/opt/aeppic/data"
Source Data
It is possible to define a second data directory as a read-only fallback to source data from.
"sourceDataDir": "/opt/aeppic/stage/my-test"
This is used to specify an alternate data store which base data and changes are read from and combining them with data from the main dataDir.
- First it tries to load the base file from the
${dataDir}/base
folder. - If no system file is present it will try to load from
${sourceDataDir}/base
folder. - Then it will load all changes from
${sourceDataDir}/changes
- Then it will load the standard changes from
${dataDir}/changes
This feature can be used to start a new version of the server with existing data without altering it. It is used by the Staging feature
Source Policies
When loading from the specified source data a policy can be applied to limit loading
"sourceDataPolicy": "latest",
"sourceDataPolicies": {
"latest": {...},
"myPolicy": {...}
}
Features (Feature Flags)
Feature Flags play a pivotal role in managing the evolution of software (See article). They are designed for use in continuous delivery pipelines, allowing new features to be introduced and deprecated progressively. This approach provides flexibility, enabling developers to test new features in a controlled environment before a full-scale rollout.
It's important to distinguish Feature Flags from Preferences. While they might seem similar, they serve different purposes. Preferences are settings that can be adjusted globally or on a per-user basis to alter the default behavior of certain features. They provide users with the ability to customize their experiences based on their needs or preferences.
For instance, let's imagine we're adding a new "Dark Mode" feature to our application. Using a Feature Flag, we can control the release of this feature, enabling it for a small subset of users or certain environments first for testing and gradual rollout.
The Preference, on the other hand, is what allows users to turn on the "Dark Mode" once it's rolled out. Some users might prefer the traditional "Light Mode", so they don't change this setting. Others, especially those who often work at night or in low-light conditions, might prefer the "Dark Mode" and choose to enable it in their user settings.
In this way, Feature Flags control the deployment and access to the feature, while Preferences allow users to customize their interaction with it.
Naming rules
When defining a feature namespace or name, certain characters, such as : or @, are not permitted.
The aeppic client uses the aeppic namespace to manage its Features module. If you're developing custom applications on aeppic, you can extend this with namespaces that are specific to your application. Please note that the aeppic server reserves the namespaces beginning with @aeppic/*, which are used to control specific server features.
To avoid namespace conflicts, it's crucial to ensure your namespace is unique. We recommend prefixing it with your registered npm organization name or web domain. This approach not only ensures uniqueness but also helps in identifying the source of the feature.
json
{
...
"features": {
"aeppic": {
"feature-name": {
"enabled": boolean // Optional. Defaults to `true`
"description": string // Optional
"options": object // Optional
"deprecated": boolean // Optional
"deprecationNotice": string // Optional
}
},
"@aeppic/api": {
},
"<namespace>": {
...
},
}
...
Static Files
The Web Server has to serve a number of static files across multiple locations. This is configured in server.json under statics
. It has three parts:
- always
- development
- production
json
{
...
"statics": {
"always": [...],
"development": [...],
"production": [...]
}
}
always
is always applied, development
is served when NODE_ENV
is set to development
and production
is the alternate used when NODE_ENV
does not match development.
Each section defines urls matched to files located relative to the aeppic server directory.
Either it exposes a complete folder under an url prefix:
json
{ "prefix": "/aeppic/libs", "folder": "node_modules/aeppic/libs" }
or it exposes a single file with a specific path
json
{ "url": "/main.js", "file": "dist/dev/main.js" }
Some files are not relevant for service worker caching (e.g .js.map
files). Those files or folders can be exempted with "offline": true
. They still need to be listed to be available on demand via normal static content serving.
json
{ "url": "/main.js.map", "file": "dist/dev/main.js.map", "offline": false }
Advanced options
The option refreshStaticList
can be set to every-call
which causes the statics/_list
API call to always run a full filesystem enumeration on every call. This is a rather expensive operation and should only be done during development.
Not having this option set to every-call
means the server will not realize when assets change in the filesystem after startup in the statics/_list
call, but changed files and new content will still be served correctly as it only affects the list. Since the static files usually do not change after boot this option is only really helpful for development.
json
{
"refreshStaticList": "every-call",
"always": ...
}
Note
This section is very important for the offline client behavior. All static path rules are exposed via the API for a service worker to fill its offline static file cache.
Caching
The caching rules by default use ETag behavior for all files with no other form of caching allowed. Improved caching should be handled by a service worker to ensure consistent assets.
Single-Page Application (SPA)
The template to server when falling through all static paths is configured in the html
section.
json
{
...
"html": {
"index": "./server/index.template.html"
}
}
It is possible to further fine-tune the template instead of just replacing it fully. For that you can override stylesheet sets and their import order and scripts to be loaded.
json
{
...
"head": {
"stylesheets": {
"sets": {
"icons": [
"/libs/fontawesome-pro-5.15.4-web/css/svg-with-js.min.css"
],
"legacy": [
"/aeppic/content/aeppic.min.css"
],
"next": [
"/dist/main.css"
],
"loading": [
{ "file": "/dist/loading.css", "inline": true }
]
},
"load": ["legacy", "icons", "loading"]
},
"scripts": {
"load": [
"/aeppic/libs/less/dist/less.min.js",
]
}
},
},
...
}
Trusted Proxy
The trusted proxy is required to ensure X-Forwarded-For headers are trusted and used for origin IP resolution in logs and for certain security decisions from outside sources.
WARNING
The IP address of the accessing client is used not just in logging but also when making decisions requiring trust. You have to ensure the proxy is actually trusted.
A simple ip
json
{
"trustedProxy": "10.0.0.1"
}
An array of simple ips or net masks
json
{
"trustedProxy": ["10.0.0.0/8"]
}
Trusted Admin Ips
Certain administrative calls are only allowed from trusted ips (after taking into account the Trusted Proxy settings).
Can be a simple IP (IPv6 is supported) or net masks (see Trusted Proxy)
json
{
"trustedAdminIps": ["127.0.0.1", "::1" ,"::ffff:127.0.0.1"]
}
Upload Limits
In order to limit uploads speeds and to avoid server overloading, an upload throttle can be configured. It will throttle all incoming streams to a total maximum desired throughput rate. This maximum rate applies to all uploads at the same time which are not specifically exempted. So 100 MBit/s could mean 5 * 20 MBit/s.
Uploads made by certain users, ip-ranges, admins can be exempted and are configured by default. External services can thus continue uploading at maximum available speed.
The throttle is configured in Megabits per second (MBs with base 10, NOT MiB/s base 2).
json
{
"bandwidth": {
"uploadThrottleInMegabitsPerSecond": 100,
"exemptAdminsFromThrottle": true,
"exemptIpsFromThrottle": ["10.1.0.0/16", "127.0.0.1", "::1" ,"::ffff:127.0.0.1"],
"exemptAccountsFromThrottle": ["some-account"]
}
}
Too Busy
When a server is very busy, certain measures might be necessary to limit access to the system.
The configuration section new_session_limits
manages the behavior of accessing the main web application. It is triggered whenever the website/app is opened in a new browser tab. So any web request to the website itself where the URL route is not handled by a static file response or the api.
The child concurrent_uploads
when enabled allows monitoring the number of concurrent uploads and cause the new session limit to be activated.
It triggers whenever more files are uploaded than configured and:
- The call is not originating from an exempted ip range
- The caller is not already authenticated by a previous visit
- The caller is authenticated but does not match the configured exceptions
then the configured html page is returned.
The default busy.html
tries a reconnect after a few seconds and triggers a full reload once the limit has been lifted.
Note that
maxUploads
is independent of bandwidth actually used.
json
{
"toobusy": {
"new_session_limits": {
"concurrent_uploads": {
"enabled": true,
"page": "/busy.html",
"options": {
"maxUploads": 20,
"doNotLimitAdmins": true,
"doNotLimitAccounts": [],
"doNotLimitIps": ["10.1.0.0/16", "127.0.0.1"]
}
}
}
}
}
Descendant TimeStamp tracking
By default the server tracks whenever a document is changed (which is reflected in the document.t field). In many cases it would be helpful to know the latest change of a descendant document in the tree. E.g. when is the last time a descendant of e.g. an app was changed.
This is not without some overhead if we did this for every document and updated the time in every ancestor. Therefore we only track this if the document is a descendant of a certain form.
The tracking itself is done via in memory tracking and not automatically injected into the document. The information can be read via the query API by setting dts: true
as part of the query options.
Configuration
json
{
"features": {
"@aeppic/server": {
"descendantTimestamps": {
"enabled": true,
"description": "Track descendant timestamps",
"options": {
"fullTrackWhenDescendantOf": [ "system", "apps" ],
"fullTrackWhenDescendantOfForm": [ "application-form", "form-folder" ],
"minTrackDepth": 2
}
},
}
}
}
Threaded InMemory API
By default the in memory api runs in a separate worker thread, this can be configured or disabled via
Configuration
json
{
"features": {
"@aeppic/server": {
"inMemoryApiOnThread": {
"enabled": true,
"options": {
"allowAttachDebugger": true,
"enabledDebuggerSignal": "SIGUSR2"
}
},
...
},
...
}
}
By default that also allows to attach a debugger onto this thread when SIGUSR2 is received by the main process.
In addition when using a threaded api calls to the export function (for example used by upload garbage collecting ) it is important to not do this synchronously on the model thread as that would block the thread for running queries. If the export was run via the api proxy we thus inject pauses into the enumeration which is configured via the following settings.
json
{
...
"apiThread": {
"iteration": {
"blockSize": 50,
"blockDelay": 10
}
},
...
}
This will configure the thread proxy to enumerate in blocks of 50 (sent via the thread worker bus to the calling thread) and then delay by 10ms.
Startup
In some circumstances it is convenient to start a machine in a test environment but to ignore some recent changes. The syntax that can be used is the one supported by parse-duration
Beware: Use ONLY in test environments with a full backup available !
Changes at the cutoff date and later could be overwritten, so going backwards in little steps is advised.
json
{
"startup": {
"ignoreRecentChanges": "-30d"
}
}
Ignore system/app changes
After loading the base files from the base/active path changes are being applied on top of that. With ignoreAppSystemChanges
it is possible to ignore any changes that try to modify changes located in the /apps or /system folders.
This is especially helpful when transporting new apps or system versions to an existing machine where no changes were supposed to have happened anyway.
Beware that any changes that should be kept would have to be transported backwards.
Also any system configured like this, will reset any system or apps changes with every reboot
json
{
"startup": {
"ignoreAppSystemChanges": false
}
}
File Reference statistics
With trackFileReferenceStats
the in memory model can be configured to track file usage statistics in the fileReferences.stats
file located in the data directory.
Details
When talking about files this refers to file fields inside documents.
md
[My File][file]
[file]: File
json
{
"stats": {
"trackFileReferenceStats": true
}
}
The statistics can be used by aeppic-files
(located in the servers bin
folder) to create information about which uploaded binary files are still in use or which files referenced files are missing.
The statistics file is a SQLite3 database and is defined by file-reference-stats.ts
Custom Headers
WARNING
Custom headers can override certain security sensitive behavior in browsers. Any changes here should only be made after thinking through the implications.
It is used for example to allow serving service worker scripts from dynamic file field which require a certain HTTP header.
Allow custom headers for _content
Sometimes there is a need to return specific HTTP headers when reading data from the _content endpoint reading dynamic data. The headers
section allows to configure this.
Example to allow Service Worker File Content
In case a service worker script is desired to be loaded from dynamic content (e.g using the Aeppic.Content.getSrc
) the script would not be able to be registered at a higher scope (See here) without being hosted at an appropriate root level by the web server or including the Service-Worker-Allowed
header. That header is not a standard header, but is defacto required and supported by all modern browsers.
In order to allow dynamic loading of service workers there is a includeServiceWorkerAllowed
setting.
E.g with the configuration below With it you can configure a matching rule to allow including it with requested content. When a /.../_content
GET request comes in which includes the query parameter headers=service-worker
then the two headers are added if the document is a descendant of the ancestors defined in ancestorIsAnyOf
.
This header setting
json
{
"headers": {
"service-worker": {
"include": [
[ "Service-Worker-Allowed", "/" ],
[ "Content-Type", "text/javascript" ]
],
"require": {
"ancestorIsAnyOf": ["system", "bd95c380-480e-41f9-88dd-790f143ab0f8"]
}
}
}
}
Dynamic component Lookups
Designs and controls are dynamic components which are looked up by name. The lookup can be configured to use a different strategy or to cache the results.
The full strategy is explained in Design and Controls
Option | Description |
---|---|
strategy | The strategy to use for lookup. Can be full , system . Default is classic for now |
cache | Whether to cache the results. Default is true |
fallback | Whether to use the previous strategy. Default is false |
json
"control-lookup": {
"enabled": true,
"description": "Lookup controls with an improved search algorithm.",
"options": {
"strategy": "classic",
"cache": true,
"fallback": false
}
}
Note: If the feature is turned on and set to classic strategy, it will use the new and classic strategy in parallel and compare the results before returning the classic result. Discre
Option | Description |
---|---|
strategy | The strategy to use for lookup. Can be full or classic . Default is classic for now |
cache | Whether to cache the results. Default is true |
json
"design-lookup": {
"enabled": true,
"description": "Lookup designs with an improved search algorithm.",
"options": {
"strategy": "classic",
"cache": true
}
}
SystemD default configuration
ini
[Unit]
Description=Aeppic Server
After=multi-user.target
# After=network-online.target
# Requires=network-online.target
RestartSec=5
StartLimitIntervalSec=0
[Service]
# SyslogIdentifier must be unique
SyslogIdentifier=aeppic-server
Type=simple
Environment=NODE_ENV=development
# Environment=NODE_ENV=production
Environment=AEPPIC_SERVER_CONFIG=/etc/opt/aeppic/server.json
# Usually configured to /opt/aeppic/server/<version>
WorkingDirectory=/opt/aeppic/server
ExecStart=/usr/bin/env node --optimize-for-size --max-old-space-size=16384 --nouse-idle-notification --expose-gc bin/aeppic-server
Restart=always
[Install]
# WantedBy=network-online.target
WantedBy=multi-user.target
Full Default configuration
json
{
"hostname": null,
"port": 80,
"ip": "0.0.0.0",
"https": {
"enabled": false,
"ip": null,
"port": 443,
"key": null,
"cert": null,
"allowHttpNoRedirect": true
},
"trustedAdminIps": ["127.0.0.1", "::1" ,"::ffff:127.0.0.1"],
"trustedProxy": "10.1.4.1",
"toobusy": {
"new_session_limits": {
"concurrent_uploads": {
"enabled": false,
"page": "/busy.html",
"options": {
"maxUploads": 20,
"doNotLimitAdmins": true,
"doNotLimitAccounts": [],
"doNotLimitIps": ["10.1.0.0/16", "127.0.0.1"]
}
}
}
},
"bandwidth": {
"uploadThrottleInMegabitsPerSecond": null,
"exemptAccountsFromThrottle": [],
"exemptAdminsFromThrottle": true,
"exemptIpsFromThrottle": ["10.1.0.0/16", "127.0.0.1", "::1" ,"::ffff:127.0.0.1"]
},
"dataDir": "/opt/aeppic/data",
"sourceDataDir": "",
"sourceDataPolicy": "latest",
"sourceDataPolicies": {
"latest": {
},
"example-2022": {
"description": "Old data only, using new system",
"filters": {
"only-data-from-year-2022": {
"description": "Only import data from up to end of 2022, but run with newest system data",
"max(modified.at)": "2022-12-31T23:59:59.999Z",
"except": {
"system": {
"any(a)": ["system","apps","a56a9acf-73b8-4f8d-9627-9d1698bb2af2"],
"reason": "system, apps, and preferences (a56...) should be included to test the new system against the old data"
}
}
}
}
}
},
"logLevel": "info",
"app": {
"title": "aeppic"
},
"services": {
"forwardFullUrls": true,
"portMapping": {
"59000": {
"ignore": true,
"description": "service-thumbnails",
"urlEnv": "SERVICE_THUMBNAILS_URL"
},
"59007": {
"ignore": true,
"description": "service-zip",
"url": "http://test:59007"
}
}
},
"stats": {
"trackFileReferenceStats": true
},
"security": {
"yubico": {
"clientId": "",
"secretKey": "",
"trustedKeys": []
},
"impersonation": {
"fixed": ""
},
"anonymous": {
"canChange": true,
"canAccessContent": true
},
"cookieSecret": "3venTZW7yQgqmU8XZfBdrZ7ZKqcvZgyV",
"metrics": {
"accessIp": ["78.47.22.84"]
}
},
"apis": {
"converter": "http://10.1.4.67",
"printToPdf": "http://10.1.4.8"
},
"caching": {
"client": {
"allowContentCaching": true
}
},
"startup": {
"ignoreAppSystemChanges": false
},
"features": {
"aeppic": {
"forms-rendering-experimental": {
"enabled": true,
"description": "Experimental forms rendering using forms views.",
"options": {
"layoutId": "6c39f2cc-6059-42b2-8796-5c803899367c"
}
},
"meta-modified": {
"enabled": true,
"description": "Do not updated modified automatically on every change. Exclude updates that do not affect data e.g move/changeForm and allow the developer to opt out on save (Useful for migrations etc)."
},
"worker-writer": {
"enabled": true,
"description": "Server Adapter that submits changes via worker to indexed db and then to the server."
},
"commands-rights-verification": {
"enabled": true,
"description": "Allow verification that the user has the rights to execute a command."
},
"commands-rights-verification-ignore-allExecutionRights": {
"enabled": false,
"description": "List of users to ignore 'aa3ba2de-4b65-4152-ab1d-655e21de9efd' for",
"options": {
"accounts": []
}
},
"websockets": {
"enabled": true,
"description": "Enable websockets to connect to the server",
"options": {
"authenticate": true
}
},
"service-worker": {
"enabled": false,
"description": "Enable service worker support. This will allow the app to work offline and load faster on repeat visits.",
"options": {
"mode": "offline",
"noopServiceWorker": "/noop-service-worker.js",
"offlineServiceWorker": "/sw.js",
"customServiceWorker": "",
"scope": "/"
}
},
"offline": {
"enabled": false,
"description": "Enable offline support (requires service-worker). This will allow the app to work offline.",
"requires": ["service-worker"],
"options": {
"encryptSignatureOnly": true,
"hash": "SHA-256"
}
},
"experimental-rust-api": {
"enabled": false,
"description": "Enable the experimental Rust API. This will allow the app to use the Rust API for some operations.",
"options": {
"apiUrl": "http://127.0.0.1:8963/api"
}
},
"opentracing": {
"enabled": false,
"description": "Enable OpenTracing support. This will allow the app to trace requests.",
"options": {
"serviceName": "aeppic"
}
},
"control-lookup": {
"enabled": true,
"description": "Lookup controls with an improved search algorithm.",
"options": {
"strategy": "classic",
"cache": true,
"fallback": false
}
},
"design-lookup": {
"enabled": true,
"description": "Lookup designs with an improved search algorithm.",
"options": {
"strategy": "classic"
}
},
"editable-data-validation": {
"enabled": true,
"description": "Enable editable validated access to editable documents data fields."
}
},
"@aeppic/server": {
"inMemoryApiOnThread": {
"enabled": true,
"options": {
"debugger": {
"allow": true,
"activationSignal": "SIGUSR2",
"port": 9230,
"ip": "127.0.0.1",
"wait": false
}
}
},
"descendantTimestamps": {
"enabled": true,
"description": "Track descendant timestamps",
"options": {
"fullTrackWhenDescendantOf": [ "system", "apps" ],
"fullTrackWhenDescendantOfForm": [ "application-form", "form-folder" ],
"minTrackDepth": 2
}
},
"websockets": {
"enabled": true,
"description": "Enable websockets to connect to the server",
"options": {
"authenticate": false,
"batchSize": 200
}
},
"businessRules": {
"enabled": true,
"options": {
"batchSize": 20,
"batchDelay": -1,
"writeDelay": 5
}
},
"model": {
"enabled": true,
"options": {
"writeBatchSize": 100,
"autoFlushInterval": 100,
"minFlushInterval": 500
}
}
}
},
"apiThread": {
"iteration": {
"blockSize": 500,
"blockDelay": 5
}
},
"html": {
"index": "./server/index.template.html"
},
"head": {
"stylesheets": {
"sets": {
"icons": [
"/libs/fontawesome-pro-5.15.4-web/css/svg-with-js.min.css"
],
"legacy": [
"/aeppic/content/aeppic.min.css"
],
"next": [
"/dist/main.css"
],
"loading": [
{ "src": "/dist/loading.css", "inline": true }
]
},
"load": ["legacy", "icons", "loading"]
},
"scripts": {
"load": [
"/aeppic/libs/less/dist/less.min.js",
"/aeppic/libs/gildas-lormeau-zip.js-563fe1d/WebContent/zip.js",
"/aeppic/libs/gildas-lormeau-zip.js-563fe1d/WebContent/zip-ext.js"
]
}
},
"headers": {
"service-worker": {
"include": [
[ "Service-Worker-Allowed", "/" ],
[ "Content-Type", "text/javascript" ]
],
"require": {
"ancestorIsAnyOf": ["system", "bd95c380-480e-41f9-88dd-790f143ab0f8"]
}
}
},
"statics": {
"development": [
{ "url": "/main.js", "file": "dist/dev/main.js" },
{ "url": "/main.js.map", "file": "dist/dev/main.js.map", "offline": false }
],
"production": [
{ "url": "/main.js", "file": "dist/min/main.js" },
{ "url": "/main.js.map", "file": "dist/min/main.js.map", "offline": false }
],
"always": [
{ "url": "/aeppic/libs/less/dist/less.min.js", "file": "node_modules/aeppic/libs/less/dist/less.min.js" },
{ "url": "/aeppic/content/aeppic.min.css", "file": "node_modules/aeppic/content/aeppic.min.css" },
{ "url": "/dist/loading.css", "file": "dist/loading.css" },
{ "url": "/favicon.png", "file": "webroot/manifest/favicon-32x32.png" },
{ "prefix": "/libs/fontawesome-pro-5.15.4-web/svgs", "folder": "webroot/libs/fontawesome-pro-5.15.4-web/svgs" },
{ "prefix": "/libs/pdfjs/web", "folder": "webroot/libs/pdfjs/web" },
{ "url": "/libs/pdfjs/aeppic.css", "file": "webroot/libs/pdfjs/aeppic.css" }
]
}
}