v12 dashboard: FortiWeb

[kryonsx]

Part of: #2696 · Needs first: #2697 (click-through to the Log Explorer and the value/count table)
Related: #1976 (parsers and rules for this integration, owned by the detection team)

Goal

Ship a built-in FortiWeb dashboard that shows, at a glance, how much FortiWeb is sending, whether it is still sending, and what kinds of events they are. The heart of it is the list of FortiWeb logs grouped by event types (log ID) (widget W7): click one and the Log Explorer opens on exactly those logs.

Where the data comes from

Integration (catalog name)	FORTIWEB
Data type	firewall-fortiweb
How the logs arrive	The FortiWeb web application firewall sends syslog (a syslog policy enabled under 'config log syslogd', per the setup guide) over UDP or TCP port 7018 to the UTMStack forwarder, which passes each line unchanged; each line is followed by key=value pairs (date=... log_id=20000008 ... type=attack ... action=Alert_Deny ...).
Parser	definitions/filters/fortinet/fortiweb.yaml
What dataSource holds	The IP address the syslog message came from: the FortiWeb itself, or a relay in between (the forwarder's own host name when the sender is 127.0.0.1). Proof: collectors/forwarder/collector/syslog/handler.go resolveRemoteAddr (lines 21-33) and readLoop (line 79, TCP), listener.go lines 264-266 (UDP); handleMessage (lines 178-182) sends DataType: inst.DataType and DataSource: msgDS.DataSource; log-input/ingest/server.go applyDefaults (lines 158-160) only writes 'unknown' when it is empty; fortiweb.yaml never writes dataSource. So W6 is 'Logs by FortiWeb IP'. The appliance serial number is in log.deviceid (raw key device_id, so that name depends on the underscore change).
Grouped by	log.logid (event types (log ID))

Why log.logid: FortiWeb puts an 8-digit log ID (log_id=20000008) in the header of every log. It classifies the message: 200000xx are attack types, one ID per attack 'main type' (for example 20000008 Signature Detection); 30000000 is traffic; 0000xxxx, 1000xxxx, 1100xxxx and 1999xxxx are event messages (FortiWeb 7.0 Log Reference, 'Log ID numbers' and 'Attack logs by main type, subtype & ID'). The key=value step (fortiweb.yaml lines 29-32) stores it as text in log.logid, because the engine strips the underscore from the key name (go-sdk utils.SanitizeField). Running the parser's step code over 451 example lines from the FortiWeb 7.0 log reference gave log.logid on 450 (the one miss is a text-extraction artifact). It is a code. The readable attack name exists in the log (main_type="Signature Detection"), but the parser cuts it at the first space and keeps the opening quote ('"Signature'), and two attack types collapse into one ('"Custom' for both Custom Signature Detection and Custom Access), so it cannot be used. log.logid is the least-bad choice; the front end should show the ID-to-name lookup below. The name depends on the underscore change: with an engine that keeps underscores it becomes log.log_id.

Typical values: 20000001 Allow Method (HTTP method violation), 20000002 Protected Hostnames, 20000003 Page Access, 20000005 Parameter Validation, 20000006 Black IP List, 20000007 URL Access, 20000008 Signature Detection (SQL injection, cross-site scripting, known exploits...), 20000009 Custom Signature Detection, 20000010 Brute Force Login, 20000014 DoS Protection, 20000017 File Upload Restriction, 20000018 GEO IP, 20000021 Custom Access, 20000022 IP Reputation, 20000026 HTTP Protocol Constraints, 20000037 Machine Learning, 20000041 Bot Detection, 30000000 Traffic (one HTTP transaction), 10000016 Administrator login (successful or failed), 10000017 Administrator login failed

Widgets

Standard layout from the parent issue; rows W2, W3 and W9 onward are specific to this integration.

#	Title	Shown as	Query	Why
W1	Total logs	number	logs: count	All FortiWeb logs in the selected time range.
W2	Attacks detected	number	logs: count; filter log.type is one of [attack]	Every attack log, blocked or only logged.
W3	Attacks blocked	number	logs: count; filter log.type is one of [attack]; action is one of [Alert_Deny, Period_Block, Return_403_error]	Attacks FortiWeb stopped (deny, temporary block of the client, or a 403 page). 'Alert' means logged but let through.
W4	Alerts	number	alerts: count	Alerts raised from FortiWeb logs.
W5	Log volume over time	area chart	logs: count over time	Shows gaps and spikes in what the FortiWeb appliances send.
W6	Logs by FortiWeb IP	bar chart	logs: top 10 values of dataSource	dataSource is the sending appliance's IP address, so this shows which FortiWeb is busiest or silent.
W7	Top event types	value and count table	logs: top 25 values of log.logid; filter log.logid exists	The main list: FortiWeb log IDs by count, shown with the attack or event name from the lookup; click one to open those logs.
W8	Event types over time (top 5)	line chart	logs: count over time, one line per value of log.logid (top 5); filter log.logid exists	When each of the five most common log IDs happened; attack waves show up as spikes.
W9	Attacks by server policy	bar chart	logs: top 10 values of log.policy; filter log.type is one of [attack]; log.policy exists	Which protected application (server policy) is attacked most.
W10	Attacks by source country	bar chart	logs: top 10 values of origin.geolocation.country; filter log.type is one of [attack]; origin.geolocation.country exists	Where attacks from public IPs come from (private addresses get no country).
W11	Top attacking IPs	bar chart	logs: top 10 values of origin.ip; filter log.type is one of [attack]; origin.ip exists	Clients that trigger the most attack logs; candidates for blocking.
W12a	Top attacked URLs	bar chart	logs: top 10 values of log.httpurl; filter log.type is one of [attack]; log.httpurl exists	Which pages attacks target. Field name depends on the underscore change (see caveats).
W12b	Actions taken on attacks	bar chart	logs: top 10 values of action; filter log.type is one of [attack]	How FortiWeb handled attacks: Alert (only logged) versus Alert_Deny, Period_Block and other blocking actions.
W13	Alerts by rule	bar chart	alerts: top 10 values of name	Which FortiWeb detection rules fire most.
W14	Alerts by severity	bar chart	alerts: top 50 values of severity	Split of FortiWeb alerts into low, medium and high.
W15	Latest logs	table of latest logs	logs: latest 20 records; columns @timestamp, dataSource, log.logid, log.type, action, origin.ip, log.policy, log.msg	The newest raw records; log.msg gives the attack message (for example '"HTTP Method Violation"').

Fields used and where they come from

• log.logid: FortiWeb log ID, text with leading zeros. Name depends on the underscore change (becomes log.log_id). Examples: 20000008, 30000000, 00001002. Source: kv step lines 29-32 on log.kvMessage (key 'log_id', underscore stripped by go-sdk utils.SanitizeField)
• log.type: Log type. Examples: attack, event, traffic. Source: kv step lines 29-32 (key 'type'; unquoted in the FortiWeb 7.0 log reference examples)
• action: Attack logs: Alert (logged, allowed), Alert_Deny, Period_Block, Return_403_error. Event logs: login, edit, add, del. Examples: Alert_Deny, Alert, edit. Source: rename log.action -> action lines 60-63
• origin.ip: Client IP of the HTTP request (attack and traffic logs; event logs have none). Examples: 185.220.101.45. Source: rename log.src -> origin.ip lines 72-75
• target.ip: Protected server (virtual server) IP. Examples: 10.101.0.1. Source: rename log.dst -> target.ip lines 64-67
• target.port: Destination port (number). Name depends on the underscore change: the rename expects log.dstport. Examples: 80, 443. Source: rename log.dstport -> target.port lines 68-71 (key 'dst_port' with the underscore stripped)
• log.policy: FortiWeb server policy (the protected application), with its quote marks. Examples: "FortiWeb_Policy_Default_AutoTest". Source: kv step lines 29-32 (key 'policy'); no quote trim in the filter
• log.httpurl: Requested URL with query string, with its quote marks. Name depends on the underscore change (becomes log.http_url). Examples: "/autotest/site_publishing_helper/login_check/0". Source: kv step lines 29-32 (key 'http_url')
• log.msg: Message text with its quote marks. Present on attack and traffic logs; missing on event logs (see parser issues). Examples: "HTTP Method Violation", "Brute Force Login Violation". Source: delete log.msg lines 35-37; rescue grok lines 40-48; tail-strip grok lines 51-57
• origin.geolocation.country: Country name of a public client IP (UTMStack's own lookup). Examples: Germany. Source: dynamic com.utmstack.geolocation, source origin.ip, destination origin.geolocation, lines 119-124
• log.maintype: Attack main type, cut at the first space with the opening quote kept; not used. Name depends on the underscore change (main_type). Examples: "Signature, "Allow. Source: kv step lines 29-32 (key 'main_type')

Watch out for

• Field names that depend on the underscore change: log.logid (W7, W8, W15; raw key log_id, becomes log.log_id), log.httpurl (W12a; http_url becomes log.http_url) and target.port (the rename at lines 68-71 expects log.dstport; the log.dest_port rename at lines 85-88 does not match FortiWeb's dst_port, so target.port would stay in log.dst_port). They were kept because nothing better exists: no parser step sets an attack-type or URL field under a name of its own. Re-point these widgets when the engine change lands. All other fields used here are safe (checked by re-running the samples with underscores kept).
• The filters assume the value style shown in the FortiWeb 7.0 log reference: type=attack and action=Alert_Deny without quote marks. The parser never removes quotes, so if a firmware quotes these values the filters must include the quotes.
• Values are stored exactly as sent. Quoted single-word values keep their quote marks, so W9 labels read "FortiWeb_Policy_Default_AutoTest" and W12a labels "/login.php", quotes included; clicking still works because the Log Explorer filter uses the stored value. Values with spaces are cut at the first space (the key=value step splits on spaces), which is why the attack names main_type, sub_type and signature_subclass are not used.
• W3 counts attack logs whose action is Alert_Deny, Period_Block or Return_403_error, the blocking actions seen in the FortiWeb 7.0 log reference examples. Other blocking actions (for example a redirect) may use other words; W12b lists the real values, so check it on a live device and extend W3 if needed.
• actionResult is not used: the parser sets 'blocked' only when action equals 'Deny' (ignoring case, line 104), and FortiWeb writes Alert_Deny, Period_Block or Return_403_error instead, so it was never set on any of the 451 reference examples.
• Event logs have no origin.ip (the admin's address is only inside msg) and lose log.msg (see parser issues), so W9 to W12b filter on attack logs. Coverage measured by running the parser step code over 451 FortiWeb 7.0 reference examples: log.logid 450/451, log.type 450/451, action on 416 of 422 event logs and all 25 attack logs, origin.ip and log.policy on all attack and traffic logs, log.msg on 1 of 422 event logs.
• origin.geolocation exists only for public client IPs (plugins/geolocation/geolocate.go IsLocal). If FortiWeb sits behind a load balancer or CDN, src is that device and W10 and W11 show it instead of the real clients.
• Every field depends on the line starting with '' (grok at lines 21-26). If a relay strips the priority, log.kvMessage is never written, the key=value step fails and the log keeps only raw.
• The alert widgets may stay mostly empty: 6 of the 7 FortiWeb rules compare action with lowercase words ('deny', 'alert', 'alert_deny', 'block') while FortiWeb writes 'Alert_Deny' and 'Alert' (the comparison is case sensitive), and several read log.attack_type or log.message, which the parser never writes. Only the file-upload rule (text inside log.msg) can fire as written.
• Parser behavior was checked by running the EventProcessor step code (grok, kv, trim, rename, add, delete) with go-sdk v1.1.26 regular expressions, CEL conditions and event conversion over the FortiWeb samples and the reference examples. The geolocation lookup was simulated. Not tested on a live UTMStack, and the engine version in the v12 image comes from a CI variable, so it could not be confirmed.

Parser problems found while designing this

These are not dashboard work, but they limit what the dashboard can show. They belong to #1976; raise them there rather than working around them in the dashboard.

• log.msg is lost whenever msg is the last key: 'delete log.msg' (lines 35-37) runs first, then the rescue grok (lines 40-48) needs another key=value after msg. FortiWeb event logs end with msg, so log.msg was missing on 421 of 422 reference event examples. Fix: let the rescue pattern accept the end of the line.
• No quote handling: the key=value step (lines 29-32) splits on spaces and nothing removes quotes. Multi-word values are cut and keep the opening quote ('"Signature' for main_type, '"United' for srccountry), and single-word values keep both quotes (log.policy, log.httphost, log.httpurl, log.subtype). The readable attack type is therefore unusable, and 'Custom Signature Detection' and 'Custom Access' both become '"Custom'. Fix: a quote-aware key=value split, or rescue groks for main_type, sub_type, signature_subclass, srccountry and http_agent, plus quote trims.
• actionResult 'blocked' requires action to equal 'Deny' (line 104), but FortiWeb's blocking actions are Alert_Deny, Period_Block and Return_403_error, so actionResult is never set. Fix: map those values to 'blocked' and 'Alert' to a 'logged' or 'allowed' result.
• target.port relies on the engine stripping underscores (rename of log.dstport, lines 68-71); the alternative rename of log.dest_port (lines 85-88) does not match FortiWeb's key dst_port. With the underscore change target.port disappears. Fix: rename log.dst_port as well.
• 'subtype' (event and traffic logs) and 'sub_type' (attack logs) both land in log.subtype today; with the underscore change they split into two fields. Give the attack subtype its own explicit name.

How to build it

Before you start: parser field names are changing while the parsers are updated for the engine's new underscore handling (see "Field names are about to move" in #2696). Check every field in this issue against the v12 parser at that moment and against real logs, and build against what you find.

1. Add definitions/dashboards/integration-fortiweb.yaml. Keep it in the top folder: the test that checks shipped dashboards (TestEveryShippedDashboardDefinitionIsValid) only reads the top folder.
2. Start from the file below; it follows the table above and passes the same checks as the backend (domain.Spec.Validate).
3. W7 is a value/count table: a category query shown with chart type table. It already renders; v12 dashboards: groundwork for integration dashboards (click-through to the Log Explorer, value/count table, Agents dashboard fix) #2697 makes its rows clickable and its headers readable.
4. Load real logs from this technology on a v12 test server (or replay samples) and check every widget before opening the pull request.

Starting dashboard file

# Dashboard version v1.0.0
#
# System-owned default dashboard for the FortiWeb integration, seeded by
# backend/modules/dashboards/repository/dashboard_bootstrap.go.
# Field names come from definitions/filters/fortinet/fortiweb.yaml.
# Verify every widget against real logs before shipping.
name: "FortiWeb"
description: "What FortiWeb is sending: volume, event types, and the activity worth a look."
widgets:
  - layout: { x: 0, y: 0, w: 3, h: 2 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: metric
      metric:
        agg: count
    config:
      __builder:
        chartType: metric
        title: "Total logs"

  - layout: { x: 3, y: 0, w: 3, h: 2 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: metric
      metric:
        agg: count
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
    config:
      __builder:
        chartType: metric
        title: "Attacks detected"

  - layout: { x: 6, y: 0, w: 3, h: 2 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: metric
      metric:
        agg: count
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
        - field: "action"
          op: in
          value: ["Alert_Deny", "Period_Block", "Return_403_error"]
    config:
      __builder:
        chartType: metric
        title: "Attacks blocked"

  - layout: { x: 9, y: 0, w: 3, h: 2 }
    spec:
      dataset: alerts
      dataType: "firewall-fortiweb"
      chart: metric
      metric:
        agg: count
    config:
      __builder:
        chartType: metric
        title: "Alerts"

  - layout: { x: 0, y: 2, w: 8, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: time
      metric:
        agg: count
    config:
      __builder:
        chartType: area
        title: "Log volume over time"

  - layout: { x: 8, y: 2, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "dataSource"
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Logs by FortiWeb IP"

  - layout: { x: 0, y: 6, w: 6, h: 6 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "log.logid"
      filters:
        - field: "log.logid"
          op: exists
      limit: 25
    config:
      __builder:
        chartType: table
        title: "Top event types"

  - layout: { x: 6, y: 6, w: 6, h: 6 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: time
      metric:
        agg: count
      dimension: "log.logid"
      filters:
        - field: "log.logid"
          op: exists
      limit: 5
    config:
      __builder:
        chartType: line
        title: "Event types over time (top 5)"

  - layout: { x: 0, y: 12, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "log.policy"
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
        - field: "log.policy"
          op: exists
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Attacks by server policy"

  - layout: { x: 4, y: 12, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "origin.geolocation.country"
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
        - field: "origin.geolocation.country"
          op: exists
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Attacks by source country"

  - layout: { x: 8, y: 12, w: 4, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "origin.ip"
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
        - field: "origin.ip"
          op: exists
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Top attacking IPs"

  - layout: { x: 0, y: 16, w: 6, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "log.httpurl"
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
        - field: "log.httpurl"
          op: exists
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Top attacked URLs"

  - layout: { x: 6, y: 16, w: 6, h: 4 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "action"
      filters:
        - field: "log.type"
          op: in
          value: ["attack"]
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Actions taken on attacks"

  - layout: { x: 0, y: 20, w: 6, h: 4 }
    spec:
      dataset: alerts
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "name"
      limit: 10
    config:
      __builder:
        chartType: bar
        title: "Alerts by rule"

  - layout: { x: 6, y: 20, w: 6, h: 4 }
    spec:
      dataset: alerts
      dataType: "firewall-fortiweb"
      chart: category
      metric:
        agg: count
      dimension: "severity"
    config:
      __builder:
        chartType: bar
        title: "Alerts by severity"

  - layout: { x: 0, y: 24, w: 12, h: 6 }
    spec:
      dataset: logs
      dataType: "firewall-fortiweb"
      chart: table
      metric:
        agg: count
      limit: 20
      columns: ["@timestamp", "dataSource", "log.logid", "log.type", "action", "origin.ip", "log.policy", "log.msg"]
    config:
      __builder:
        chartType: table
        title: "Latest logs"

Done when

• definitions/dashboards/integration-fortiweb.yaml is merged to release/v12.0.0 and go test ./modules/dashboards/... passes in backend/.
• With FortiWeb logs flowing on a v12 test server, every widget shows data. A widget that stays empty while logs arrive means a wrong field or value: fix it, don't ship it.
• Clicking a row, bar or line point opens the Log Explorer on the same logs, and the Log Explorer count matches the widget.
• A time range with no logs shows empty states, not errors.
• A screenshot of the finished dashboard is attached to this issue.