A few months back, my good friend Flemming Riis and I found a fundamental security vulnerability in the IBM Tivoli Storage Manager (TSM) client, while researching IBM TSM’s handling of authentication ("Node ID" and "Node Password") and unsafe implementations of TSM, which we covered in a few blogposts: Backdoors and data compromise via Backup Systems (in danish only) and Protecting your secrets.
We couldn't believe our own eyes when we, in very little time, found a pretty important – and incredibly trivial – security vulnerability in the TSM product.
We then initiated a Responsible Disclosure process with the IBM Product Security Incident Response Team (PSIRT), which I will describe further below, as it illustrates how important it is for researchers to set requirements and insist on deadlines being met.
The vulnerability
An attacker can basically use built-in tools to read the TSM "Node ID" and an obfuscated value of the "Node Password" – merely as a regular user on any Windows system with the TSM client installed, such as a Terminal/Citrix server.
Above is shown the specific permission setting in the registry, where basic users are allowed Read permission to highly sensitive information (the "Node Password" value).
With this information, the attacker can restore previously backed up files – no matter the NTFS permissions etc. – to another machine, on which the attacker has administrative permissions.
This could be the attacker’s private laptop, connected to the network and with a TSM client installed.
Apart from unauthorized access to files (documents, spreadsheets, configuration files etc.), the attacker could, for example, gain access to everything from the SAM database (password hashes) to sensitive registry-values, and potentially clear text passwords for service accounts.
A malicious insider is an obvious threat in regards to this vulnerability, giving the attacker the possibility of Information Disclosure, Privilege Escalation and Credential Theft.
The Responsible Disclosure process
In the following, I'll give my description of the process working with the IBM PSIRT.
In my eyes, this shows the importance of setting strict requirements for vendors, especially regarding deadlines. Ultimately for the customer’s (and therefore for the supplier’s own) sake.
03-03-2017 (DD-MM-YYYY) Vulnerability reported via online form to IBM PSIRT, with the initial deadline set to 10-03-2017 (7 days). Prior to that, IBM had to test and confirm the reported vulnerability - or we would publish our findings. The second and ultimate deadline was set to 03-05-2017 (60 days).
07-03-2017 IBM PSIRT confirmed the reception of our inquiry with an Advisory ID.
07-03-2017 We told IBM PSIRT, that we intended to collaborate and eventually releasing information on the vulnerability when a workaround or patch was out, but no later than 03-05-2017.
10-03-2017 IBM PSIRT replied that our "submission has been verified and will be addressed in a future release", and their Product Team was "hoping" to solve the problem "late 3rd Quarter 2017/early 4th Quarter 2017".
11-03-2017 We confirm to IBM PSIRT that they met the first deadline (10-03-2017). We stick to the release deadline 03-05-2017, but at the same time indicate, that we are willing to give them more time, if they collaborate and answer some questions on criticality (CVE) etc. We ask about the possibility of the vulnerability also affecting non-Windows systems.
12-03-2017 IBM PSIRT informed us that our questions were forwarded to the developers.
20-03-2017 We ask IBM PSIRT for a response from the developers.
22-03-2017 IBM PSIRT reply with extensive answers to our questions. The developers say a fix is complex and they request more time.
05-04-2017 We make it clear to IBM PSIRT that we haven’t seen sufficient commitment to a release date for a patch or workaround. We give IBM PSIRT a new deadline of 03-06-2017, giving them an additional 30 days to provide a workaround or patch. We also tell them, that we hope they'll protect their customers before that. We once again encourage them to roll out a simple workaround regarding the assignment of permissions in the registry.
21-04-2017 We ask IBM PSIRT for confirmation that they received our inquiry of 05-04-2017 and ask for a status update. We also ask for a response to the question (from 11-03-2017) regarding non-Windows systems.
24-04-2017 IBM PSIRT replied that they approached the Product Team on 21-04-2017 and will await their response.
27-04-2017 IBM PSIRT apologizes for the delay. They aim at releasing mitigation before the end of May in the form of a Security Bulletin. An actual patch (fix) would be released as previously planned (3rd/4th Quarter 2017). Furthermore, IBM PSIRT tells us that the vulnerability Flemming Riis and I found independently apparently had already been reported to IBM in September 2016, and that all researchers would be recognized in the final Security Bulletin. Therefore, Kęstutis Gudinavičius is also listed under Acknowledgement, but we have never communicated.
23-05-2017 We ask IBM PSIRT for a status update and a date for the publication of the workaround and Security Bulletin. We (once again) ask them for a response regarding non-Windows systems as well.
23-05-2017 IBM PSIRT replied that they are releasing mitigation before the end of May. They further confirm that they will check and include all supported versions of TSM on any platform.
01-06-2017 We ask the IBM PSIRT for information on where they published the mitigation and Security Bulletin.
01-06-2017 IBM PSIRT replied that they published the mitigation and Security Bullletin the day before, and they provided us with this link.
The main thing to take from this is, that the previous researcher apparently didn't put pressure on IBM to fix the problem. Therefore, the vulnerability could survive and thrive until we stumbled upon it. I find this very problematic.
During this period (and probably years before), IBM TSM customers have been vulnerable to this relatively trivial attack vector. This is not rocket science, and a simple solution requires only minimal effort.
During the last months, I have come to think of the missing standards for Responsible Disclosure timelines, bug bounty programs, etc., but that is a topic for another time.
Recommendations
I recommend that anyone who uses the IBM TSM product:
implements the workaround on all relevant systems (primarily servers with TSM backup client installed), and
limits network access to TSM servers, so only relevant systems (the ones that need backup and restore) can communicate with the TSM backend servers (standard TCP 1500).