Veeam Decoy Project

Let’s start from the beginning: security and backup.

Today, unfortunately, ransowmare attacks are on the rise, and defending against them is an increasingly difficult challenge.

If backups used to be considered as something not really important, perhaps useful only in case of any storage damage, today they have become the last resort to keep our data safe.

For this reason, one of the main targets during a cyber attack is the backup infrastructure: if threath actors succeed in taking it down, the road to ransom payment will be straight downhill.

News of collaborations and product integrations between large data protection and security vendors are now a daily occurrence, most recently the one between Veeam and Palo Alto Network Cortex XSIAM/XSOAR.

All this brings home to us how important it is to focus on the security of all systems, including backup infrastructure.

One of the several best practices recommended by Veeam, for example, is to try to make its components as anonymous as possible.

Assigning backup servers and repositories a name that cannot be identified with their role can be a first attempt to avoid making just about everything so easy for any malicious attackers.

Another method for attempting to identify and perhaps slow down an ongoing attack is to use honeypots: traps, decoys used to attract threat actors and draw them out.

The honeypot is a component that simulates the production system, possibly with the same applications, but with data that is not real.

In the case of Veeam Data Platform, the idea might be to create a VBR server that acts as a honeypot, perhaps even equipped with working backups.

Of course, this might require a not inconsiderable effort, because we would have to use sacrifiable, non-production systems, with the only purpose of attracting malicious attackers and having our anomaly detection software detect instrusion or tampering attempts on the honeypot.

A more simple option is the one developed by the open source Veeam Decoy project.

This system simulates multiple Veeam and Windows services, such as Veeam Backup Server services, Veeam Hardened Repository, Veeam Windows Repository, Veeam Backup Enterprise Manager, SSH, RDP, Netbios.

It supports the use of multiple network cards, so each service can be associated with a specific VLAN, so it is ready for a realistic attack scenarios using lateral movement tactics (TA0008).

The system doesn’t receive any incoming traffic, so any connection attributable to the use of discovery tactics (TA0007) should represent an intrusion attempt.

This tool can be downloaded as an OVA appliance (compatible only with vSphere 8.0 or higher) or installed on a minimal Rocky Linux.

The console comes with a very simple yet comprehensive interface where we can manage the status of decoy services, associated network interfaces, and view real-time ports in use and connection logs on each specific service.

All captured connection attempts, including information such as source port, source ip, or credentials used, can be turned over to a centralized syslog or via email, so that alerting can be triggered and readily handled by a SOC.

Of course, we do not expect it to be our most effective weapon against cyber attacks, but in this battle between the two worlds it is still one more option! đź’š

Veeam 12.2 – What’s New

During this week, the long-awaited news arrived: Veeam 12.2 is finally available for download.

As previewed in this article, there are many new features compared to the previous version.

The main ones are:

  • support for Proxmox VE, with immutability on backup and cross-platform VM restore capability
  • improved Nutanix integration, including support for backup operations via Prism Central with Veeam backup for Nutanix AHV 6
  • native backup support for Mongo DB, one of the most popular NoSQL databases, including the classic explorer for granular restores
  • full support for VMware vSphere 8.0 U3 and VMware Cloud Director 10.6
  • improvement of integrations with IBM Db2 and SAP HANA
  • support for Amazon Redshift and Amazon Fsx
  • support for Microsoft Azure Data Lake and Cosmos DB

In addition to the above, some of the most interesting improved features include:

  • support for direct offload from performance tier to archive tier for all types of repositories present on-prem in the SOBR
  • CDP I/O Filter Cross Compatibility, to also support older versions (12.0 and 12.1)
  • Veeam App for Splunk, an extension that allows users of the popular software to monitor the status of the Veeam backup environment
  • introduction of two new RBAC roles, Incident API Operator and Security Administrator
  • new checks added in the Security & Compliance Analyzer
  • sure backup continuous schedule, selecting specific time windows
  • database authentication for Oracle RMAN Plugin
  • intelligent SOBR extent selection for backup of unstructured data
  • immutable snapshots integration for HPE Storage Arrays

For the full list of all new features, see the vendor’s official document here.

Enjoy! đź’š

CrowdStrike – Global Incident

Last Friday, July 19, the popular U.S. software company CrowdStrike caused a worldwide crash of Windows-based computers, impacting critical systems in banks, hospitals, transportation..resulting in a temporary disruption of daily operations.

The cause? An incorrect update of the Falcon Sensor AV/EDR platform, released as a configuration update at 04:09 UTC, which resulted in triggering a logical error on the OS resulting in a BSOD:

The content is a channel file located in the %WINDIR%\System32\drivers\CrowdStrike directory.

Channel file “C-00000291*.sys” with timestamp of 2024-07-19 0527 UTC or later is the reverted (good) version.

Channel file “C-00000291*.sys” with timestamp of 2024-07-19 0409 UTC is the problematic version.


The company subsequently released a procedure for identifying affected Windows clients, as well as a remediation plan, advising to perform a reboot and acquire the correct version of the file, on alternatively follow Microsoft’s procedure to enter safe mode and delete the affected file, or alternatively restore the system (beware of any Bitlocker Key present).

EDIT: an official recovery tool has also been released by Microsoft to automate the remediation process.

The problem has impacted not only physical PCs, but also Windows instances in the cloud. Below are some official links for remediation, such as Azure and AWS.

In any case, if the problem instances are protected with backup software, it is always possible to restore to the latest valid version.

This event reminds us of how today’s world is extremely technology-driven, and of the possible human errors that, if not limited, can lead to disastrous consequences.

VeeamON 2024 Recap

INTRO

Last week in Fort Lauderdale, Florida, VeeamON 2024 took place, as every year the most awaited and important event organized by Veeam Software.

This year’s event was particularly rich in announcements, and there was really no lack of surprises.

Lots of demos and technical sessions, though not all available for those like me who followed everything remotely.

Veeam’s vision continues to focus on data resilience through 5 main strategies: Data Backup, Data Freedom, Data Recovery, Data Security, and Data Intelligence.

WHAT’S NEW

Starting to explore Data Backup, the core part dedicated to protecting and saving data, new versions of some solutions were officially presented.

  • Oracle Linux Virtualization Manager (oVirt): native support for OLVM, a KVM-based virtualization platform, has been available already since a few weeks
  • Proxmox VE: announced a few weeks ago the compatibility with this virtualizer, during VeeamON 2024 the first demo was presented, with the Veeam solution that promises to be 3 times faster than the native backup solution. The official release is scheduled for next Q3 2024
  • VBA v7: some new features for the future version of Veeam Backup for Azure announced, including the introduction of support for Cosmos DB
  • Veeam Backup for AWS v8: new features also for the Amazon cloud backup solution, introducing, for example, support for Redshift and Fsx
  • VBM365 v8: many new features also for Veeam Backup for Microsoft 365, coming out probably next Q3 2024, including MFA for console, proxy pools, immutability for backup, restore operator audit in Veeam ONE
  • Veeam Backup for Salesforce v3: additional features for this solution as well, where support for data encryption, data archive and data pipeline will be introduced
  • K10 v7: of course, it could not be missed an overview of the new version of Kasten, which includes, among others, support for FIPS-Enabled Clusters, for Azure Blob Immutability and for VMs on Openshift

We now turn to the surprises, which, as anticipated, were not lacking. Notable among the new features announced were, without a doubt:

  • VBR server on Linux OS starting with v13, with the specific capabilities of native zero trust architecture, and support for HA of Config DB, which will add that level of resilience and automation to the software that to today was lacking
  • Enter ID Backup, a solution that will be intergrated into Veeam B&R, to protect data, such as users, groups and app registrations, from Microsoft’s cloud-based identity/access management solution (Q4 2024)
  • Mongo DB Plugin, which will increase the package of natively supported enterprise applications (Q3 2024)
  • Lenovo TruSacle Backup, which will intregrate Veeam Backup & Replication and Veeam ONE into Lenovo ThinkSystem solutions for on-premise backups


In addition, as we know, Veeam has recently expanded its range of solutions by introducing fully SaaS services, further explored at this three-day event, including:

  • Veeam Data Cloud for M365, a preconfigured Microsoft 365 backup solution with a predictable cost model (per user/unlimited space)
  • Veeam Data Cloud for Azure, native and optimized backup solution for Microsoft Azure
  • Veeam Vault, fully managed cloud storage, with flat/TB rates, including api call charges and any outbound traffic

We move on to the Data Freedom and Data Recovery strategies, which is Veeam’s ability to use its own format to move a piece of data from one platform to another, allowing it to bypass the so-called “vendor lock-in”.

In this section we can mention the announcement of more new features for the upcoming version of VRO (Veeam Recovery Orchestrator).

Regarding Data Security, that strategic component through which Veeam and its solutions help data to be resilient to increasingly frequent cyber attacks, much space was given to Coveware, a company specializing in incident response acquired by Veeam last April 2024.

In particular, the key role it can play in a Cyber Recovery phase was explored, as it offers services such as:

  • Assessment
  • Forensic Analysis
  • Identification of ransomware type and impact on the customer’s organization
  • Negotiation with cybercriminals
  • Incident remediation and documentation


Also in the area of Data Security, also worth mentioning is the new partnership with Palo Alto for SIEM integration.

Speaking of Data Intelligence, another big surprise presented was the formalization of the partnership with Microsoft for Copilot AI integration with Veeam solutions.

Finally, we must mention other improvements and developments announced on Veeam ONE, Veeam AI assistant, Linux Hardened Repository and Veeam Service Provider Console.

CONCLUSION

In short, there was a lot of news, and I’m sure there will be an opportunity to explore some of them in more detail in future posts..STAY TUNED! đź’š

Veeam – Proxmox Announcement

This week Veeam Software made the much expected announcement: support for Proxmox will be released soon.

What is Proxmox, and why so much interest behind this news?

Proxmox VE (Virtual Environment) is an open-source, KVM-based virtualizer that allows both virtual machines and container-based architectures to run.

The recent acquisition of VMware by Broadcom, and the subsequent unknowns about the future strategies of the world leader in virtualization systems, have pushed many customers to look for possible alternatives to focus on for their infrastructures.

For this very reason, the name of Proxmox has gained popularity in the recent period, so much so that even Veeam has decided to focus on developing integration with this new hypervisor.

The first official demo will be presented at VeeamON 2024 to be held in Florida next June 3-5.

If you have not yet registered you can do it here.

Enjoy! đź’š

Veeam – Wasabi Object Storage

When we talk about backup repositories in Veeam, we have to mention object storage, a technology that has been growing in popularity in recent years.

From version 12 of Veeam B&R, in fact, it is possible to directly write a backup to this type of repository.

Since version 12.1, it has also been possible to back up data stored on an object storage.

Unlike file system type storage architectures, which manage data hierarchically within directories, object storage architecture is flat, and is designed to store unstructured data, such as backups.

Specifically, the data is divided into blocks with associated metadata and unique identifiers, which are used by the system when accessing it.

The main advantages include that it can hold large amounts of data at no excessive cost, is easily scalable, and is compatible with HTTP/HTTPS and REST API protocols.

Wasabi is one of the cloud-based object storage vendors, so we can compare it to the better-known S3 from AWS or Azure Blob Storage from Microsoft.

Unlike the large vendors mentioned above, the price/TB is much lower, and there are no costs for ingress/egress traffic or API calls.

Wasabi is listed in the Veeam Ready compatibility directory as an object storage backup target (S3 compatible), and with native support for immutability (object lock) functionality.

The first thing to do to use Wasabi for our Veeam backups is to create a storage account by registering for the free 30-day trial; after that, it is possible to continue using the account in Pay As You Go or Reserved Capacity Storage mode.

Once registered and logged into the dashboard, generate a new access key/secret key pair, and create the bucket that will store our Veeam backups:

Now we can go to our Veeam B&R console, and from the main menu click on “Add Repository,” then select “Object Storage” and “Wasabi Cloud Storage”:

Once the wizard starts, enter the name we want to give on Veeam to our Wasabi repository:

Next, enter the details of the storage account and region on which we created our bucket:

At this point, enter the details of the bucket and folder to be used for our backups:

NB: for this tutorial in a lab environment the immutability flag was not enabled, but for production environments it is always recommended to use it

Finally, specify the mount server and complete the wizard:

Here is our Wasabi repository to use for our backup jobs:

Enjoy! đź’š

Veeam ONE 12.1 – Threat Center

Veeam ONE is Veeam software’s solution for monitoring virtual environments, such as vSphere, Vmware Cloud Director, Hyper-V, and data protection environments, such as Veeam Backup and Replication and Veeam Backup for Office 365.

As mentioned in a previous post, the latest VONE 12.1 release introduced the Veeam Threat Center dashboard: this tool allows us to view the overall security status of our VBRs, verifying compliance with the various best practices indicated by Veeam.

Specifically, the widgets we find are:

  • Data Platform Scorecard: shows an overall score of the health of our VBRs, defined by the parameters Platform Security Compliance, Data Recovery Health, Data Protection Status and Backup Immutability Status
  • Malware Detections: shows any malware or suspicious infections on our restore points
  • RPO Anomalies: shows objects that are out of range from the defined RPO
  • SLA Compliance Overview: highlights the percentage of achievement of our SLAs based on a period and success rate defined in the widget configuration

In order to take advantage of the potential of this dashboard, we must first add our VBR, making sure to also check the “Provide access to embedded dashboards” checkbox

Before configuration, within the VBR console the integration will not be active:

After configuration, the dashboard will be populated with the Veeam Threat Center view of Veeam ONE and other useful widgets.

Tip: when adding a VBR, pay attention to the compatibility of the licenses of the two products

https://helpcenter.veeam.com/docs/one/deployment/license_types.html?ver=120#compatibility-with-veeam-backup—replication-licenses

Enjoy! đź’š

Linux xz library vulnerability

Last Friday, a major vulnerability was reported on the xz library, used by some Linux distributions as a data compression program.

Specifically, the source code on Github was infected with malicious code properly obfuscated, allowing attackers to create a backdoor for ssh access to infected systems.

The CVE is currently listed by NIST with criticality 10.0, which is highest:

https://nvd.nist.gov/vuln/detail/CVE-2024-3094

The vulnerability, discovered almost accidentally by a Microsoft developer, is present in versions 5.6.0 – 5.6.1

Therefore, it is recommended to downgrade the xz library version on systems with this release, or to uninstall it if not in use.

Below is also the official note from Red Hat:

https://access.redhat.com/security/cve/CVE-2024-3094

WBD 2024

Today, March 31, is World Backup Day 2024!

This anniversary, which started way back in 2011, wants to make businesses as well as individuals aware of the importance of having their data backed up and safe with an annual commemoration.

If we think about any business in the world, small or large, we know for sure that every day it has to manage data to carry out its work. Personal registries, orders, payments are a few examples of activities that are indispensable for a business, activities that need to write this data in almost always digital devices, whether they are servers, storage or simple computers.

Let us now think of ourselves, our family members, our friends. Who among us does not use a smartphone or computer? Photos, videos, messages, important documents, all valuable material that we certainly don’t want to lose.

But what if the computer suddenly breaks down or our smartphone gets stolen? We would no longer be able to recover our data, unless we had backed it up first!

Well yes, for any important data, it is good practice to make at least a second copy and keep it in a safe place. Cloud, external hard drives, usb sticks are some examples of devices that can help us save our important files, a backup indeed!

Sometimes we do not realize the importance of something until we have lost it, but why take the risk?

Backup is the solution!

Microsoft OOB Updates

Microsoft released an important Out-Of-Band (OOB) update yesterday, which is an emergency fix to be installed before the upcoming April updates, for Windows Server versions 2022, 2016, and 2012 (not yet available for the 2019 version).

This update fixes a know issue that was identified in the last update in March: the problem afflicts the Active Directory Domain Controllers’ LSASS service, where a memory leak during kerberos authentication requests can cause the service to crash and unexpectedly reboot the server.

Microsoft recommends installing the update immediately in case your system falls into the described case scenario.

https://learn.microsoft.com/en-us/windows/release-health/status-windows-server-2022#3271msgdesc

Below are the reference KBs for the specific versions:

Windows Server 2022: KB5037422
Windows Server 2019: KB5037425
Windows Server 2016: KB5037423
Windows Server 2012 R2: KB5037426

EDIT: The update for Windows Server 2019 was realeased yesterday.