Tag: Security

Veeam Decoy Project

Posted on by BAdmin

Let’s start from the beginning: security and backup.

Today, unfortunately, ransowmare attacks are on the rise, and defending against them is an increasingly difficult challenge.

If backups used to be considered as something not really important, perhaps useful only in case of any storage damage, today they have become the last resort to keep our data safe.

For this reason, one of the main targets during a cyber attack is the backup infrastructure: if threath actors succeed in taking it down, the road to ransom payment will be straight downhill.

News of collaborations and product integrations between large data protection and security vendors are now a daily occurrence, most recently the one between Veeam and Palo Alto Network Cortex XSIAM/XSOAR.

All this brings home to us how important it is to focus on the security of all systems, including backup infrastructure.

One of the several best practices recommended by Veeam, for example, is to try to make its components as anonymous as possible.

Assigning backup servers and repositories a name that cannot be identified with their role can be a first attempt to avoid making just about everything so easy for any malicious attackers.

Another method for attempting to identify and perhaps slow down an ongoing attack is to use honeypots: traps, decoys used to attract threat actors and draw them out.

The honeypot is a component that simulates the production system, possibly with the same applications, but with data that is not real.

In the case of Veeam Data Platform, the idea might be to create a VBR server that acts as a honeypot, perhaps even equipped with working backups.

Of course, this might require a not inconsiderable effort, because we would have to use sacrifiable, non-production systems, with the only purpose of attracting malicious attackers and having our anomaly detection software detect instrusion or tampering attempts on the honeypot.

A more simple option is the one developed by the open source Veeam Decoy project.

This system simulates multiple Veeam and Windows services, such as Veeam Backup Server services, Veeam Hardened Repository, Veeam Windows Repository, Veeam Backup Enterprise Manager, SSH, RDP, Netbios.

It supports the use of multiple network cards, so each service can be associated with a specific VLAN, so it is ready for a realistic attack scenarios using lateral movement tactics (TA0008).

The system doesn’t receive any incoming traffic, so any connection attributable to the use of discovery tactics (TA0007) should represent an intrusion attempt.

This tool can be downloaded as an OVA appliance (compatible only with vSphere 8.0 or higher) or installed on a minimal Rocky Linux.

The console comes with a very simple yet comprehensive interface where we can manage the status of decoy services, associated network interfaces, and view real-time ports in use and connection logs on each specific service.

All captured connection attempts, including information such as source port, source ip, or credentials used, can be turned over to a centralized syslog or via email, so that alerting can be triggered and readily handled by a SOC.

Of course, we do not expect it to be our most effective weapon against cyber attacks, but in this battle between the two worlds it is still one more option! 💚

CrowdStrike – Global Incident

Posted on by BAdmin

Last Friday, July 19, the popular U.S. software company CrowdStrike caused a worldwide crash of Windows-based computers, impacting critical systems in banks, hospitals, transportation..resulting in a temporary disruption of daily operations.

The cause? An incorrect update of the Falcon Sensor AV/EDR platform, released as a configuration update at 04:09 UTC, which resulted in triggering a logical error on the OS resulting in a BSOD:

The content is a channel file located in the %WINDIR%\System32\drivers\CrowdStrike directory.

Channel file “C-00000291*.sys” with timestamp of 2024-07-19 0527 UTC or later is the reverted (good) version.

Channel file “C-00000291*.sys” with timestamp of 2024-07-19 0409 UTC is the problematic version.


The company subsequently released a procedure for identifying affected Windows clients, as well as a remediation plan, advising to perform a reboot and acquire the correct version of the file, on alternatively follow Microsoft’s procedure to enter safe mode and delete the affected file, or alternatively restore the system (beware of any Bitlocker Key present).

EDIT: an official recovery tool has also been released by Microsoft to automate the remediation process.

The problem has impacted not only physical PCs, but also Windows instances in the cloud. Below are some official links for remediation, such as Azure and AWS.

In any case, if the problem instances are protected with backup software, it is always possible to restore to the latest valid version.

This event reminds us of how today’s world is extremely technology-driven, and of the possible human errors that, if not limited, can lead to disastrous consequences.

Veeam ONE 12.1 – Threat Center

Posted on by BAdmin

Veeam ONE is Veeam software’s solution for monitoring virtual environments, such as vSphere, Vmware Cloud Director, Hyper-V, and data protection environments, such as Veeam Backup and Replication and Veeam Backup for Office 365.

As mentioned in a previous post, the latest VONE 12.1 release introduced the Veeam Threat Center dashboard: this tool allows us to view the overall security status of our VBRs, verifying compliance with the various best practices indicated by Veeam.

Specifically, the widgets we find are:

  • Data Platform Scorecard: shows an overall score of the health of our VBRs, defined by the parameters Platform Security Compliance, Data Recovery Health, Data Protection Status and Backup Immutability Status
  • Malware Detections: shows any malware or suspicious infections on our restore points
  • RPO Anomalies: shows objects that are out of range from the defined RPO
  • SLA Compliance Overview: highlights the percentage of achievement of our SLAs based on a period and success rate defined in the widget configuration

In order to take advantage of the potential of this dashboard, we must first add our VBR, making sure to also check the “Provide access to embedded dashboards” checkbox

Before configuration, within the VBR console the integration will not be active:

After configuration, the dashboard will be populated with the Veeam Threat Center view of Veeam ONE and other useful widgets.

Tip: when adding a VBR, pay attention to the compatibility of the licenses of the two products

https://helpcenter.veeam.com/docs/one/deployment/license_types.html?ver=120#compatibility-with-veeam-backup—replication-licenses

Enjoy! 💚

Linux xz library vulnerability

Posted on by BAdmin

Last Friday, a major vulnerability was reported on the xz library, used by some Linux distributions as a data compression program.

Specifically, the source code on Github was infected with malicious code properly obfuscated, allowing attackers to create a backdoor for ssh access to infected systems.

The CVE is currently listed by NIST with criticality 10.0, which is highest:

https://nvd.nist.gov/vuln/detail/CVE-2024-3094

The vulnerability, discovered almost accidentally by a Microsoft developer, is present in versions 5.6.0 – 5.6.1

Therefore, it is recommended to downgrade the xz library version on systems with this release, or to uninstall it if not in use.

Below is also the official note from Red Hat:

https://access.redhat.com/security/cve/CVE-2024-3094

Veeam 11a Patch – EOS

Posted on by BAdmin

As you may know, as of March 1, 2024, several outdated versions of Veeam products have gone into EOS (End of Support).

Examples of the most used products include Veeam B&R 11 and Veeam ONE 11:

To see the full list of Veeam product lifecycle visit the following link:

https://www.veeam.com/product-lifecycle.html

This week, a little surprisingly, a cumulative patch for Veeam Backup & Replication V11a was released:

https://www.veeam.com/kb4245

This update comes especially to those customers who due to usage requirements need to maintain compatibility with older hypervisors ( for example VMware vSphere/Esxi 5.5).

The patch contains some product fixes, and also some security fixes of third-party components included in the software, such as VDDK, OpenSSL, liblz4, zlib and Putty:

https://www.veeam.com/kb4245

Important note: If you decide to install this patch, you will no longer be able to upgrade to V12.1, but will have to wait for the release of the next minor update V12.2 (expected in the second half of 2024).

So if you are at V11 and have no compatibility issues with the rest of the infrastructure, the advice is to upgrade to the latest V12.1 version, taking advantage of the many added features immediately.

Below is one last link that may be useful when planning upgrades, the upgrade-path link for Veeam B&R.

https://www.veeam.com/kb2053

Enjoy! 💚

Veeam v12.1 – Security and Compliance Analyzer

Posted on by BAdmin

In a previous post, we went to explore the new and more interesting features of Veeam B&R version 12.1.

In this post we will go into more detail about the tool that allows us to keep an eye on the status of our backup infrastructure: the Security and Compliance Analyzer.

INTRODUCTION

When we design and implement our backup infrastructures, paying attention to security rules is now a must.

There are a number of general considerations that help us harden our servers, as well as many best practices that should be applied to our backups.

The new Security and Compliance Analyzer tool allows us to have just such a simple and intuitive overview of the implementation of these best practices on our backup server.

Let’s go through its functionality in detail.

THE TOOL

Access to the tool is clearly visible in the main bar of the Veeam Console:

As anticipated earlier, the checks are divided into two sections, “Backup Infrastructure Security” and “Product Configuration“.

BACKUP INFRASTRUCTURE SECURITY

As we can see, it is concerned with checking the implementation of certain best practices defined for the Windows operating system hosting our backup server.

  • The first settings that are recommended are to disable those services that are considered critical because they allow remote interaction with our server, which are “Remote Desktop,” “Remote Registry” and “Windows Remote Management“.

  • Then we move on to the rarely considered Windows Firewall: the best practice is to keep it active always, going to work with the inbound and outbound rules as needed. Note: Veeam B&R automatically creates the firewall rules necessary for its components to communicate with each other.

  • It is then recommended to disable the “WDigest credentials caching” and “Web Proxy Auto-Discovery service” features to prevent credential or MITM-type attacks.

  • The next check is on deprecated versions of SSL and TLS, such as SSL 2.0, SSL 3.0, TLS 1.0 and TLS 1.1, which should also be disabled.

  • As for potential script-based malware attacks, good practice is trying to limit them by disabling the “Windows Script Host“.

  • Returning to deprecated protocols, SMBv1 is also among those to be disabled, as it is affected by numerous vulnerabilities. Note: As of Windows Server 2016, it is disabled by default.
  • The last protocol to be disabled is the “Link-Local Multicast Name Resolution” to limit spoofing and MITM attacks.

  • Finally, security on the SMBv3 protocol is tested, checking that settings to prevent NTLMv2 relay type attacks are enabled.

PRODUCT CONFIGURATION

We now move on to controls on the software-side settings.

The strategies and configurations that Veeam recommends are obviously focused on preserving our backups.

  • MFA for the VBR console: since v12, multi-factor authentication can be enabled on the backup console.

  • Immutable or offline media: to protect backup files, we recommend using at least one repository with the data immutability feature enabled or media that can be disconnected from the network, such as tape or rotated drives.

  • Password loss protection: a setting in Veeam Enterprise Manager that allows us to decrypt our backup data in case the encryption password is lost.

  • Domain or non-domain?: Veeam recommends that we leave our server, and the other infrastructure components, at Workgroup. Note: In case we want to use join with AD, it is good practice to create a management domain dedicated exclusively to the backup environment.

  • Email notification: always remember to enable email notifications, it is essential to keep track of the outcome of backups and other events happening in the system.

  • 3-2-1 rule: the golden rule advises us to have at least 3 copies of the data (including the original data, so two backup copies), on at least 2 different media and 1 offsite copy. Note: The rule has now evolved into 3-2-1-1-0, where the second 1 is the offline/immutable copy, and the 0 indicates the need to implement an automated validation procedure for our backups, and error-free during verification testing.
  • Reverse Incremental: is the method that produces more read and write operations on our repository, to be abandoned in favour of the standard incremental.

  • Unknown Linux servers: in case we need to add linux servers to our backup infrastructure, it is recommended to trust them manually rather than automatically.

  • Configuration Backup: as a best practice, the configuration backup should be saved to a repository external to the backup server itself. Note: from v12.1 you can select an immutable object storage repository for this task as well.

  • Proxy traffic encryption: if our virtual proxies use network transport mode, encryption (NBDSSL) is recommended.

  • Physical Hardened repository: to reduce the attack surface, the hardened type repository should reside on a physical server (and with local disks) instead of a virtual one.

  • Network traffic encryption: to enable secure communication in our backup network, both to the Internet and to private networks, it is recommended to globally enable encryption in the general software settings.
  • Linux authentication: best practice recommends that we do not use password-based authentication for our linux servers, but enter SSH through the use of the public-private key pair, preventing brute force and MITM type attacks.

  • Backup services: it is recommended to use “Local System” as the account for our Veeam services.

  • Configuration backup encryption: it is recommended to use encryption on Veeam’s configuration backup as well, for more secure management of sensitive data in the DB.

  • Password rotation: control is over the credentials of the various components added to our backup infrastructure and the encryption password, which should be changed at least once a year.

  • Hardened repository access: as a best practice, SSH on this type of repository should be disabled.

  • S3 object lock type: this check verifies that the immutability set on the S3s added on Veeam is Compliance type (not editable) and not Governance type (editable), going against any policies on data handling (e.g., GDPR) and of course the security of effective immutability of backups.

  • Backup encryption: especially if our backups are saved to a cloud repository, it is a good idea to enable encryption at the individual job level.

  • Latest updates: it is recommended to keep the Veeam B&R software updated to the latest release/patch.

IMPLEMENTATION

To facilitate the implementation of most of these best practices, Veeam has provided on its KBs a powershell script that fixes all 11 points related to backup infrastructure and 2 points related to product infrastructure, while the settings that need custom setups (such as, for example, setting up our mail server, choosing a repository, users for which to enable MFA, etc.) are obviously in the hands of the backup administrator to be configured manually.

Below is the link to download the script: https://www.veeam.com/kb4525

USE

The tool can be used both interactively and automatically.

It is possible, in fact, to set up a report with daily scheduling and emailing.

Finally, it is also possible to exclude one or more parameters from the controls by marking them as “suppressed“.

CONCLUSION

We will never tire of repeating how important security is, especially for backups, which are our last defense against loss or corruption of our data. This improved tool is a good starting point to help us keep things under control.

We conclude the post by reposting other useful links regarding security and best practices, with the hope that the Security and Compliance Analyzer will also be increasingly developed and improved according to the evolving guidelines.

https://helpcenter.veeam.com/docs/backup/vsphere/security_guidelines.html?ver=120

https://bp.veeam.com/security

https://go.veeam.com/rs/870-LBG-312/images/veeam-security-checklist.pdf

https://go.veeam.com/rs/870-LBG-312/images/veeam-security-best-practices-2022.pdf

https://community.veeam.com/cyber-security-space-95/hardening-veeam-12-server-the-definitive-checklist-4255

Enjoy! 💚