CVE-2024-45519 incident response — patching a live Zimbra mail server mid-compromise with a custom Perl fix

Hilong Ecuador's production Zimbra mail server was owned. Cryptominer eating CPU, reverse shell in a persistence crontab, unauthorized SSH keys in root's authorized_keys, outbound traffic to command-and-control servers on obscure ports. The company found out because email throughput had degraded — the initial call was "our email is slow", the reality was much worse.

Triage — the first two hours#

Rule zero of IR: do not reboot. A reboot loses volatile evidence (running processes, network connections, mapped memory).

The first two hours were pure observation:

• ps auxf to enumerate every running process, with parent-child chains.
• netstat -tunapl to list every listening port and every established connection.
• ss -tunp cross-check.
• crontab -l for every user with a shell, ls -la /etc/cron.*.
• find / -name authorized_keys 2>/dev/null — audit every SSH trust.
• last -30 and wtmp review for unexpected logins.
• ausearch --start today for auditd events.

Findings, in the order they surfaced:

• A binary at /tmp/.cache/kdevtmpfsi pegging 380% CPU. Textbook Monero cryptominer.
• A reverse shell attached to a listener on 91.x.x.x:4444 originating from a process named kinsing.
• A crontab entry * * * * * /tmp/.cache/kdevtmpfsi under root, re-launching the miner if killed.
• A root SSH key in /root/.ssh/authorized_keys matching no known engineer.
• A modified /etc/rc.local re-adding the SSH key on every boot.

The entry point#

CVE-2024-45519, a critical Zimbra postjournal RCE disclosed in September 2024. The compromise timeline matched public exploit availability. The attacker had root within an hour of Zimbra being scanned.

Eradication#

Order of operations:

• Isolate the network at the switch level — not at the server. Cutting the switch port prevents the attacker from noticing we are cleaning up and reacting.
• Snapshot the disk — full raw image to an external drive for forensic preservation. This is the evidence bundle that later goes to the client's lawyer if needed.
• Kill the miner and reverse shell — kill -9 on the identified PIDs.
• Remove the persistence — delete the malicious crontab entries, the modified rc.local, the unauthorized SSH keys.
• Rotate every credential — root password, every service account, every SSH key, every API token stored on the server. Assume the attacker has all of them.
• Patch Zimbra to the CVE-fixed version — but Zimbra's official patch was not yet available for the client's OS version.

The custom Perl patch#

Zimbra's postjournal handler is a Perl script that accepts a message body over a local Unix socket. The CVE was an input sanitization flaw that let an attacker inject shell metacharacters into a downstream system() call.

The official fix was pending. The client's mail server needed to be back online today, not next week.

I wrote a 40-line Perl patch that intercepts the postjournal input, strips shell metacharacters, rejects any input matching the CVE's exploit signature, and logs the attempt. It is not a proper fix — a proper fix goes into the codepath that does the system() call. But it closes the exploitable surface and it works.

The patch was applied, Zimbra was restarted, the network isolation was lifted, and mail flow resumed. Total downtime after eradication: 12 minutes.

The official Zimbra patch was applied 4 days later when it shipped. The custom Perl patch was removed at that point.

Hardening#

The server was owned once. The goal now is to make owning it a second time expensive.

• fail2ban on SSH, Zimbra web mail, IMAP, POP3. Any client that fails auth 5 times in 10 minutes gets a 24-hour ban.
• firewalld replacing the previous default iptables. Explicit allow rules for mail ports, deny everything else.
• SSH key-only auth — password login disabled entirely.
• Root SSH disabled — an attacker who steals a key still has to escalate.
• auditd rules for filesystem writes to /etc, /var/spool/cron, /root/.ssh.
• Weekly SOC review — a scheduled log review of auth failures, unusual processes, outbound connections. The client is on a small retainer for this.

What the client got#

• Mail server back online in the same day.
• Malware fully eradicated with a preserved forensic image.
• Every credential rotated.
• Weekly SOC monitoring for early detection of re-compromise.
• A written incident report describing the compromise, the actions taken, and the residual risks.

What the client learned#

Zimbra is a target. Any mail server with public IMAP/webmail exposure is a target. The right posture is: patch aggressively, monitor auth failures continuously, keep backups off-server, and assume the box will be owned eventually — plan for that.

The engagement continues as a monthly retainer.