Libraesva ESG v5.1: Release Notes

Libraesva ESG

Main features of version 5.1

  • Users with quarantine action "release-only" or "none" cannot manage white and black lists in the UI anymore
  • Cluster: added replicate transport option for each domain (previously known as WAN cluster or Geo-Cluster option)
  • Any trusted network defined by the administrator can be configured to accept only email originated by internal domains
  • Outbound email traffic from Microsoft 365 and G Suite allows only email originated by internal domains (i.e. relay table)
  • New administrative REST API (/api/v2), stateless, JSON/HAL based, with support for Impersonation and OTP
  • Quarantine digest report: email addresses of other recipients are not listed for privacy protection
  • More granular management of the privacy password to view sensitive data
  • Message details now includes all URIs contained in the message and forensic data about them
  • Show asynchronous job status and failures to administrator on the Web UI
  • DMARC aggregated reporting can be enabled by the administrator
  • Advanced Audit log which contains all the details of the modified data

Version 5.1.17 (Aug 29, 2023)

Security

  • ClamAV: upgrade to version 0.103.10

Version 5.1.16 (Aug 21, 2023)

Security

  • Quicksand: add RAR archive reconstruction on sanitization
  • ClamAV: upgrade to version 0.103.9

Version 5.1.15 (Jun 27, 2023)

Improvements

  • System Upgrade: show more details on failure

Bug fixes

  • Microsoft 365: case insensitive username check on login

Version 5.1.14 (Apr 8, 2023)

Security

  • Savapi: update to version 4.15.16
  • Bitdefender: update to version 3.2.2

Improvements

  • Learn Actions: always update Adaptive Trust Engine when learning as good or bad

Bug fixes

  • SubmitToLabs: when submitting as bad, use learn-as-bad instead of forget
  • Quarantine reports: allow actions from quarantine report for read-only admins with “mark for relase” permissions
  • ESG 5.2 upgrade: pre-install the upgrade file for user convenience

Version 5.1.13 (Apr 3, 2023)

Improvements

  • Audit log: speed up logging during massive updates

Bug fixes

  • Removed duplicated default/relay configuration
  • Search: fixed export of search results when using advanced filters
  • System Preferences: added validation of NTP server address
  • Digest report: show links of configured actions for “Quarantine” users
  • Report: fixed MTA Statistics report using advanced timing
  • System Upgrade: keep local DNS resolved running during system upgrade

Version 5.1.12 (Mar 13, 2023)

Security

  • QuickSand: block attachments in archive when unzip size is greater than 150MB

Improvements

  • System Upgrade: require confirmation before upgrade

Bug fixes

  • File System Replica: improved stability for “Large” appliances
  • Message Viewer: hide Whitelist/Blacklist action for user with “Mark for Release” permission
  • Smarthost: fixed migration of 4.9 configuration on address-based smarthost

API

  • IMPROVED: `message-journal` reduced pagination to 150 records

Version 5.1.11 (Feb 21, 2023)

Security

  • ClamAV: upgrade to version 0.103.8

Improvements

  • TLS Certificate: validate certicate match with private key
  • Fail2ban: ignore SASL auth failed from trusted networks
  • ClamAV: increase timeout of signature update for slow connection

Bug fixes

  • Backup: fix backup of TLS Certificates
  • Database rotation: restore optimization of TxRep data
  • Graymail Management: add validation on Blocked Graymail entries
  • Top Sender Report: fix visualization of locally generated email
  • Quarantine storage extension: fix GPT partition extension
  • Search: fix “Last Hour” range to latest hour
  • Report: increase timeout of PDF generation
  • Message Details: fix popup of Dictionary rules triggered
  • User Quarantine schedule: automatically order delivery and fixed duplicated hours

Version 5.1.10 (Jan 16, 2023)

Security

  • Whaling Protection: verify SPF alignment before whaling checks

Improvements

  • Check Firewall Ports: added detection of firewall inspection on http/https
  • System Upgrade: implemented email notification after upgrade
  • System Upgrade: improved check of upgrade pre-condition
  • License Details: show contact email
  • Dashboard: show all engines as “not active” when license is expired

Bug fixes

  • Privacy Password: permit view for Read-Only Domain Admin
  • Privacy Password: fixed UI regression of version 5.1.9
  • Antispam action: fixed compile of High Spam forward rule
  • RBL Filtering: re-activated RBL check flagged as enabled
  • System notification: use absolute URL for generated email
  • Database Rotation: fixed cleanup of old Message Uris
  • Message details: show meaningful messages in pop-up when there are no results
  • Mail log: properly log to database messages with more than 300 blocked files
  • Search: fixed condition for all fields with value Yes/No

Version 5.1.9 (Dec 20, 2022)

Improvements

  • Dashboard Cluster status checks both nodes
  • FTP Backup: less disk space required before FTP upload
  • FTP Backup: execute on both nodes of cluster
  • Updated DDoS default limit for connection and message rate
  • DMARC: changed reporting organization to appliance hostname

Bug fixes

  • Quicksand: fixed sanitization of nested archives
  • Cluster: fixed option “replicate transport” for some instances
  • Cluster: fixed notification of file replica down
  • Message Details: show correct DKIM information when domain use mixed case sensitivity
  • MTA Certificate: correctly apply fallback self-signed certificates
  • FTP Backup: check disk space before execution
  • Web UI: custom pagination detail when there are no records
  • Microsoft Azure: add agent server IP to trusted IPs in firewall configuration
  • Microsoft Azure: changed keepalive settings to avoid stale support connections
  • Privacy Password: hide spurious read-only records from Domain Admin view
  • ESG4 migration: correctly configure CRM114 permission
  • ESG4 migration: check let’s encrypt configuration before migrate

API

  • ADD: added validation and other options to `/relay`
  • FIXED: avoid 500 error on `/relay` updates due to auditing issues of impersonated users
  • FIXED: avoid 500 error on `/integration/imap` and `/integration/pop3` updates due to digest report options
  • IMPROVED: increased allowed memory and timeout for `GET /messages/message-soar`

Version 5.1.8 (Nov 14, 2022)

Improvements

  • Firewall check: only use local resolver for outbound DNS port test (53)
  • User console: improved storage speed test
  • ISP Panel: add accounted email addresses count
  • License Usage: add more fields to exported data
  • Login auth test: show logs from all user providers (M365, IMAP, …)
  • Cluster auto-recover: added new auto-recover scenarios
  • Integrations: when import is triggered from the UI, forward the request to the primary node
  • Message details: contextualize WebUI notice for ignored whitelist entries
  • OpenDMARC: don’t replicate data between node so that reports are appliance based.
  • RSyslog: increase rate limit to 2048 to accomodate large appliance
  • Cluster: add the replica node to trusted IPs by default
  • Add support for Enterprise Linux 8.7

Bug fixes

  • Summary Report: date interval extremes are now included in date range
  • Improve reliability of watchdog scripts notifications
  • Microsoft 365 auth: catch server errors when the connector is badly configured
  • ESG4 migration: correcty migrate disabled antivirus customizations
  • firewall: maintain netfiltering rules during cluster setup
  • Fix audit message on TLS certificate deletion
  • Use correct DN on user import
  • Backup: added SMTP Auth users to archive
  • Antispam settings: fix saving of high spam score
  • Fail2ban: recover logging to rsyslog (as it was in ESG 4.x)

API

  • FIX: allow creation of relay based entities (regression from 5.1.4 fixed)

Version 5.1.7 (Oct 17, 2022)

Security

  • Improves RBL detection sub-domains below 3rd level
  • Quicksand: improves www-links detection in PDF
  • Update Bitdefender license
  • Relay: bounce messages sent by M365/G Suite are considered trusted (when the relative option is enabled)

Improvement

  • Whaling: improved performance of domain selection during scanning
  • Cluster auto-recovery: reduce timer interval to recover minor error faster
  • System upgrades: show estimate time in advance and use animated progress bars when upgrade is active
  • HA Proxy: locally generated email are sent directly, not through the proxy

Bug fixes

  • Outlook add-in: remove reference to a deprecated API call
  • User Manager: add JS validation for empty username on creation
  • User Manager: on user removal, cleanup job statuses owned by that user
  • UI: remove the about link which conflict with white labeling
  • Improved trial license upload
  • Passwordless: remove Libraesva from signature when white labeling is enabled

API

  • IMPROVED: added user `_permissions` property to message details
  • IMPROVED: allow domain admins to access `/api/v2` documentation page
  • FIX: add headers content-type x-requested-with to CORS headers

Changes in v5.1.6 (Oct 3, 2022)

Improvements

  • ESG 4.x migration: prevent timeout and memory exhaust on historical data restore

Bug fixes

  • System Definitions: fixed automatic update of blacklisted URL hashes
  • Release requests: fixed Domain Admin notification of release requests
  • License Usage: fixed display of accounted license
  • Login Authorized Networks: properly check login network on current IP before save or modify it

API

  • FIXED: properly apply page size on GET /message API

Changes in v5.1.5 (Sep 20, 2022)

Improvements

  • Report: show more data in reports and improve accuracy
  • Logwatch: reduce noise due to irrelevant log entries
  • Web UI: menu is collapsible from mobile device

Bug fixes

  • Message listing: shows to users email sent from restricted trusted network
  • Message logging: logs URI/attachment with very long names
  • Message details: fixed negative delay time due to timezone conversions
  • Summary report: count as outgoing email sent from restricted trusted network
  • OpenDKIM: always add restricted trusted networks as source for OpenDKIM signatures
  • Release requests: show disabled form settings to read-only users
  • Release requests: fixed selection of domain admins for a domain
  • Backup: hide non-working edit buttons to read-only admins
  • Backup: increase timeout for TAR command
  • Whitelist: allow creation of whitelist to SafeLearn network

API

  • NEW: added /report/{name} methods to query historical data (e.g. license usage /report/per_domain_mailbox_use?search=all)
  • IMPROVED: collections GET /messages order data by received date
  • IMPROVED: empty response from POST /user-message return 204 instead of 200

Changes in v5.1.4 (Sep 05, 2022)

Security

  • Add CSRF protection to all user quarantine actions
  • Message forwarding requires privacy password authorization
  • Extended quarantine links validity to 31 days

Improvement

  • Dashboard: add filtering by “last hour” and “last 3 hours”
  • Rejected message: show more message details
  • Search export: added many fields and fixed recipient numbering
  • Cluster auto-recovery: added notification when the procedure fails to auto-recover
  • Cluster: improved the auto-recovery procedure when an appliance is powered off for a very long time
  • Safe-learn web UI: describes advantages and drawback of the feature
  • Add forward compatibility for Clamd configuration of ESG > 5.1
  • System upgrades: show estimate time in advance and use animated progress bars when upgrade is active
  • API user: use automatic password generation to enforce strong password policy

Bug fixes

  • Backup: restored manual backup creation which was failed due to a missing table
  • License settings: cleanup license no longer generates memory errors
  • License settings: restore searching by email address
  • M365/G suite: resolved a race condition which may interfere with cluster replication
  • Data Loss Prevention: resolved the 503 caused by custom rule names
  • Spam Actions UI: add a better description for bounce action
  • Job status: remove stale job entries when certificate creation fails
  • Message details: after quarantine storage delete flag the message as “deleted”
  • Whitelist: fix javascript validation in some complex situation
  • Login networks: added advanced audit logs
  • Mail Encryption: auto-apply settings on save
  • Export no longer generate errors if executed by read-only or quarantine admins
  • Cofence: fixed auto learn of notified samples (regression from 5.1.3)
  • Release: show a message to the user when a message release fails
  • User Messages: show success message on completed save action
  • First boot wizard: handle non-FQDN before to initial bootstrap is completed
  • Local RBL: don’t consider quicksand sanitized as definitive threat
  • Message actions: update all leftover links in user report to the new ESG 5.x format
  • Phishing safe sites are now mirrored only by Libraesva ESG first-party mirror
  • Custom spam policy: fix javascript validation for body rules
  • LDAP import: fix manual import triggered from the WebUI
  • User manager: fixed saving of option “Can change spam settings”
  • Release requests: the notification email is optional

API

  • NEW: read-only access to saved search using “GET /saved-search”
  • NEW: added resource “/user-message” to allow email-continuity implementation
  • ADD: added property “senderIp” to message listing “GET /message”
  • ADD: added many properties to “GET /message/{id}”
  • ADD: added filtering by recipients and or relay filtering on many resource listing
  • Increased the default item per page
  • Improved speed of journal/soar calls when messages contain many URIs

Changes in v5.1.3 (Aug 08, 2022)

Improvements

  • Privacy password: show session expiration details in profile menu when permission is granted
  • Reduce web-server memory usage, especially for small appliance
  • Cluster: auto-recover cluster replication when an appliance is powered off for a very long time
  • Outlook Addin: added support for “Mark for Release” User Permission

Bug fixes

  • Clustom Antispam Rule: when adding a rule for a single domain, consider both “envelope to” and “header to” to select the domain
  • Fix web-interface redirection when DKIM is disabled
  • Release request: fix auto-removal of already managed requests
  • Content filtering: removed web-ui combination add custom header on body matches, which isn’t available in the engine
  • Mail Logs Real-Time View: better visualization for filter and reset buttons
  • ESG status: remove visualization artifact during status update
  • Mail Logs Real-Time View: stop real-time view on logout to avoid DoS
  • System Upgrade: fixed prefetch of all packages before upgrade
  • Increased timeout for quarantine report generation
  • Associate delivery to the most recent MTA message id, to avoid issues on high-traffic instances
  • Automatically apply and reload Mail Scanner configuration when changing Mail Intercept configuration and Attachment Filters
  • ESG 4.x migration: reset cache after Adaptive Truste Engine import
  • DKIM Signing: fixed generation of DKIM keys

Changes in v5.1.2 (Jul 26, 2022)

Security

  • ClamAV: upgrade to version 0.103.7

Improvements

  • Delivery details: add DSN column
  • Cluster: allow hostname change when cluster is already setup
  • Cluster setup wizard: add button to quickly copy-paste IPs
  • ESG system upgrades: better identification and feedback on Layer-7 inspection

Bug fixes

  • Adaptive Trust Engine: fixed a bug which prevented the relay age cache to be used
  • Report: pie chart used in "Per Domain Mailbox Use" defaults to licenses count not email address
  • Delivery details: delay seconds are displayed correctly
  • Reports: disabled the menu for raw data export (use export or copy instead)
  • Cluster setup: show replica IP in flashed message during first initialization
  • Database logger: fixed a bug that caused some delivery not to be bound to scanned messages
  • Local RBL: use transactional update to improve accuracy
  • Queues: ignore temporary files when computing queued message counts
  • Privacy password: fixed the flashed messages on wrong password
  • DOS prevention: when an abuser is blocked, prevent access to mail encryption portal too
  • Backup: ignore a cache table during backup
  • Cluster reload: added proper distributed reload in many pages
  • API: boolean property "user releasable" was negated in the output

Changes in v5.1.1 (Jul 20, 2022)

Security

  • URLSand: handle URI with a protocol name containing special chars

Improvements

  • Firewall checks includes cluster connectivity check
  • URI: log all URI also when message is blocked as spam
  • Quicksand: improve cache hit rate of clean PDF and Documents
  • User manager: quarantine permissions for all admin types are explicitly shown when editing users
  • Show a custom "410 gone" page for all quarantine action URL no longer available on ESG 5.x
  • File replication: send email notification also when SSH connection repeatedly timeouts
  • TLS certificates: show expiration date of active certificates

Bug fixes

  • Quicksand: a silently disarmed PDF no longer hides other files disarmed reports
  • Mail Intercept: enable/disable button is recovered
  • Email continuity: restore JS warning on invalid data
  • Advanced logging: correctly set “null” values in audit messages
  • Mail log: strip invalid UTF-8 character from security report and spam report
  • Web UI: fixed removal of Passwordless Authorized Devices
  • make sure graymail plugin is restarted on database reload
  • Quarantine settings: immediately apply per domain customization of language
  • Local RBL: make sure statistics are updated only on primary node
  • ESG 4.x migration: reset file permissions after quarantine import
  • Domain admin: allow domain names up to 256 character
  • Backup/Restore: avoid errors if quarantine files changes during a hot backup
  • Prevent creation of new domain admin password (see upgrade to version 5.1)
  • Update documentation of Whitelist and Blacklist import

Changes in v5.1.0 (Jul 11, 2022)

Security

  • TLS policy: changed default policy for new installation from "High Compatibility" to "Medium"
  • TLS policy: "Medium" policy requires TLS encryption for SMTP Authentication (as in strict policy)
  • TLS policy: "Strict" policy disables TLS renegotiation
  • Postfix: prevent DoS when multiple email to the same domain are hanging in outgoing (i.e active) queue
  • Improve detection of sender hosts in backscatter plugin
  • Web Portal Authentication: allow multiple authentication set per domain (e.g. combining G Suite with Microsoft 365 and LDAP)

Improvements

  • Quicksand and URLSand: allow default configuration at appliance level which also applies to new relays
  • Add OTP support to APIv1 login
  • Syslog: service is not restarted on "apply" when configuration is unchanged
  • Faster license usage cleanup operation through asynchronous deletion
  • Disk space monitor: notify by email when older data is removed because of critical disk threshold
  • Message details: increased readability of security reports
  • Message details: when a message is trusted, log if it's trusted by IP, by SMTP-auth or Authorized relay
  • Backup: "User Messages" configuration is now added to backup by default
  • Relay: automatic apply of engine configurations on every change
  • Cluster: the Web UI highlights all the configurations that are not replicated automatically
  • Web UI: "risky operations" are highlighted with a "danger" style (red color)
  • Mail log viewer: faster and cleaner filtering
  • Integration: improved memory consumption of Microsoft 365 and G-Suite import operations
  • User Messages: "External Warning" can now be customized
  • Asynchronous import from Microsoft 365, G-Suite and LDAP when started from the Web UI
  • Compose: show the list of available sender addresses for the logged user
  • Compose: hide main menu button when the user has no usable sender email
  • Subject tags: replaced all prefixed {tag?} with user friendlier [tag]
  • External warning: use the definition of "unusual correspondent" rather than "first time sender"

Bug fixes

  • Search page: by default search is performed as "exact match" rather than "contains" to maximize performance
  • Syslog: prevent double logging of some entries
  • G-Suite: handle Google API rate limiting to prevent issues when importing big user base
  • Wizard: allow manual configuration of Short and Long organization name
  • SNMP: only show major release in sysLocation OID
  • MTA: use hostname instead of domain for locally generated email to better distinguish generating servers
  • MTA: don't automatically add domain to envelope sender for email generated from fully-trusted IP

Breaking changes

This version introduces some changes which require your attention.

  1. Email from M365/G-Suite sent from unmanaged domains will go through additional SMTP transport checks. For incoming email,
    this will improve security since all SMTP reputation checks are always enabled. For outgoing email you have to make sure that all your domains used as "envelope sender" are listed into relay table page. Take extra care for subdomains of your relay domains, which MUST be explicitly added.
  2. Restricted user permissions for whitelist and blacklist. The web UI will honor the user's "Quarantine Action" configuration in all pages. Some users will not be able to manage whitelists and blacklists anymore if they are not explicitly given this permission. Change the "Quarantine Action" user's configuration to include whitelists and blacklists if needed. From the user manager page you may bulk update this setting for multiple users.
  3. Changed format of default subject tags. Subject prefixes added from ESG are now formatted as [Tag], while in previous versions were formatted as {Tags?}.
  4. Removed old API for ISP Panel. If this appliance is controlled by an external ESG with version 5.1, after the upgrade the appliance needs to be associated again using a new API token generated from the API v2.

Version 5.0

All upgrades from previous versions are included. See the full release notes of Libraesva ESG version 5.0.