Oxorio 2023 Security Report

In this research, we would like to present a general summary of all the work our team has accomplished over the year.

Our team of highly qualified security professionals audited a substantial total of 46,764 lines of code and identified 529 vulnerabilities. We have meticulously analyzed all the issues found in our clients’ code and wish to share with you the most critical insights in the field of security that will help determine which aspects and trends are becoming obsolete, which threats are still relevant, and what new ones will threaten the security of decentralized applications in 2024.

We will provide you with a visualization of the results of our audit reports for the entire year of 2023 and share important observations that become available for understanding only after analyzing specific figures.

We are also pleased to share that none of the projects we audited were hacked. As always, an audit from Oxorio remains a guarantee of the highest quality.

So, let’s begin with a general look at the main security development trends that have emerged throughout 2023.

Review of security trends in 2023

Let’s go through the 5 main narratives that we have managed to identify!

To more precisely describe the connection between the overall industry development trends and specific security development trends, we will first mention the general trend and then discuss the challenges that arise in connection with the development of this trend.

1. The Year of L2-Rollups Development

In 2023, we witnessed an unprecedented surge in the development of Layer 2 rollups and related solutions, all aimed at increasing throughput and developing blockchain infrastructure.

We observed the widespread adoption of the business model “Rollup as a Service”, within which we saw the emergence of very interesting market players such as Gelato, which provides infrastructure for rollups based on Polygon CDK and OPstack. Since Gelato and its competing projects utilize a considerable array of independent infrastructure solutions, a hack into this project could potentially lead to the hacking of all the rollups it hosts, resulting in catastrophic consequences. Another vector of attack here could be a breach of PolygonCDK or OPstack, especially considering that the latter’s Fraud Proof System is still under development. Regarding L2 rollups, it’s also noteworthy that rollups built on Optimium, such as Metis Andromeda and Mantle, have entirely disabled their Fraud Proof Systems.

In this context, for the protocols of the model we reviewed, the most critical period is approaching, during which they must balance between fully developing their products to completion and timely identification and remediation of architectural vulnerabilities. We see that everything remains quite raw and requires significant refinement and even more substantial investments in security. Those protocols that do not pay enough attention to this could pay a very high price for their negligence. We will be actively monitoring the changes happening here over the next couple of years.

However, alongside the positive trends, there’s a bit of disappointment - throughout the year, we’ve observed that cross-chain bridges and multichain protocols continue to be hacked almost every quarter. This is evidenced by the breaches of HYPR Network, HECO Bridge, Shibarium Bridge, Poly Network, Multichain, and several others.

From the outcomes of 2023 and experiences over the past few years, it’s clear that the industry has yet to learn how to build secure bridges. The emergence of phenomena such as Bridging Insurance also indicates that even without external hacks, bridges don’t always perform well.

This situation signifies that to attract liquidity, projects must exert effort to create truly secure methods of transmitting tokens between different chains.

2. The Year of Liquid Staking Development

The upgraded Ethereum, providing the possibility to unstake validators from the Beacon Chain, greatly spurred the development of Liquid Staking protocols, culminating with the release of Lido V2, allowing users to not only deposit but also withdraw funds from liquid staking. Since then, we’ve observed an unprecedented, rapidly growing influx of liquidity into the sector, as working with Lido, RocketPool, Mantle, and other LSD-protocols has become very simple and profitable. Such a situation attracts increased attention from both security researchers and hackers to the architecture of LSD protocols, as everyone tries to find vulnerabilities within them.

However, we also see that there remain some issues in the architecture of these protocols at the intersection of off-chain and on-chain components of architecture, the resolution of which is yet to be found. The potential use of ZKP (Zero-Knowledge Proofs) might also be a possible solution.

The importance of LSD protocols in the DeFi infrastructure can be analogously compared to the importance of systemic banks in the traditional finance world - their bankruptcy would lead to the collapse of the entire financial system. This position places enormous responsibility on the developers and auditors of these protocols. Consequently, throughout the year, we’ve observed how leading audit firms have devoted and will continue to devote the next year to a close focus on the security of Liquid Staking protocols.

3. Rising Complexity of Hacks Calls for Formal Verification & Pentesting

Throughout the year, we observed an unusually large number of new infrastructure projects, and consequently, a significant number of hacks of various Web3 infrastructures. You might recall the (second) hack of UX Balancer, issues in older versions of the Vyper compiler leading to the Curve hack, the breach in the cloud service provider used by Mixin Network, and a large number of Centralized Exchange (CEX) hacks such as Huobi, Poloniex, Remitano, among many others.

It’s clear that standard security practices aren’t very effective in these cases. For thoroughly audited projects like Curve, what was needed was not another audit but a formal verification of the language versions used in the deployed contracts, especially considering that these versions are not always the latest.

In the case of CEX hacks, to secure often very complex interfaces used by traders for multilevel interactions with different architecture components, a comprehensive penetration testing (pentesting), in addition to a couple of additional audits, could help. Pentesting allows simulating attacks on the system to identify vulnerabilities and unforeseen interaction scenarios between components, as well as testing the level of security. It not only helps detect difficult-to-find vulnerabilities but also assesses the real risks of their exploitation, thus providing an opportunity to eliminate them before hackers can exploit them. This is particularly important in Web3, where the complexity and interaction of components create many unpredictable entry points for potential attacks.

Over the next few years, seeing the trend towards more complex hacks, we predict that more advanced security analysis methods such as pentesting, fuzzing, formal verification, and others will gain popularity and increasingly be used in combination with each other.

4. Zero-Knowledge is Going Down to Mainstream

While this year cannot be called revolutionary for ZK, updates from various projects released this year demonstrate that Zero-Knowledge Proofs (ZKPs) indeed have the potential to become a strong trend in the next bull market. The number of market players is really increasing, and they are advancing applied cryptography to develop their own solutions. We’ve seen advancements like Plonky2, ZKLLVM, a large number of new SDKs and DSLs, the development of recursive STARKs, growing interest in the development of homomorphic encryption, and many more exciting things.

However, this positive trend goes hand in hand with a negative trend - the lack of security practices due to the wide variety of development approaches, the partial absence of tools for testing and security analysis of code, and an insufficient accumulation of experience concerning hacks and their prevention.

To avoid repeating the fate of DeFi, which lost hundreds of millions of dollars to hacks, ZK projects should take into account the experience of past years and timely address security issues, rather than focusing solely on developing their technological advantages.

It is clear now that over the next few years, many leading audit firms will be involved in creating a comprehensive methodology for auditing ZK projects and gaining experience so that by the time of mass adoption of ZK, they are prepared to audit large and complex scopes that include intricate, non-replicable cryptography and academic-level mathematics. This pertains to most of the Domain-Specific Languages: including Circom, Cairo, Noir, Leo, and generally most Rust-friendly languages used to write circuits today. Also, tools for various types of testing, static analysis, and other practices that help ensure security are already starting (and will continue) to appear for these languages.

5. Revolution of UniV4 pools

The update to version 4 of the Uniswap protocol introduced revolutionary changes in the field of liquidity pools, significantly expanding the possibilities and security of user interaction with decentralized finance.

The Emergence of Custom Pools: Uniswap V4 introduced custom pools, allowing users to customize pool parameters for specific strategies and needs. This means pool creators can set various types of fees, liquidity limits, and choose specific asset sets for trading. Such flexibility has allowed for higher capital efficiency and better alignment of risks and returns.

Changes in Pool Behavior Throughout Their Lifecycle: The ability to dynamically change pool parameters in response to market conditions or other factors has allowed users to fine-tune strategies and react to changes in the market environment.

We anticipate that as the functionality and complexity of pools increase, so too will the need for developing safe ways of integrating the technology into third-party projects. This includes creating standards and methodologies that minimize risks and ensure protection against potential attacks. The development of such practices, which will gain momentum throughout the next year, will be aimed at strengthening trust and stability in decentralized finance.

It is also worth mentioning Hooks — extensions or intervention points that allow changes in the system’s behavior. In the context of UniV4, safely creating and integrating hooks can significantly enhance the flexibility and security of pools, allowing, for example, automatic adaptation to changing market conditions or the implementation of new risk management strategies. Research in this area will be focused on creating universal and secure tools for developers and users.

Analysis of findings from Oxorio reports

Before proceeding with the analysis, it is worth explaining in more detail about the Security Assessment Methodology used by Oxorio Auditors.

Security Assessment Methodology

Several auditors work on a single audit, each independently checking the provided source code according to the security assessment methodology described below:

1. Project architecture review

The source code is manually reviewed to find errors and bugs.

2. Code check against known vulnerabilities list

The code is verified against a constantly updated list of known vulnerabilities maintained by the company.

3. Security model architecture and structure check

The project documentation is reviewed and compared with the code, including examining the comments and other technical papers.

4. Cross-check of results by different auditors

The project is typically reviewed by more than two auditors. This is followed by a mutual cross-check process of the audit results.

5. Report consolidation

The audited report is consolidated from multiple auditors.

6. Re-audit of new editions

After the client has reviewed and fixed the issues, these are double-checked. The results are included in a new version of the audit.

7. Final audit report publication

The final audit version is provided to the client and also published on the company’s official website.

Severity Level Reference

The following severity levels were assigned to the issues described in the report:

• CRITICAL: A bug that could lead to asset theft, inaccessible locked funds, or any other fund loss due to unauthorized party transfers.
• MAJOR: A bug that could cause a contract failure, with recovery possible only through manual modification of the contract state or replacement.
• WARNING: A bug that could break the intended contract logic or expose it to DDoS attacks.
• INFO: A minor issue or recommendation reported to or acknowledged by the client’s team.

Status Level Reference

Based on the client team’s feedback regarding the list of findings discovered by the contractor, the following statuses were assigned to the findings:

• NEW: Awaiting feedback from the project team.
• FIXED: The recommended fixes have been applied to the project code, and the identified issue no longer affects the project’s security.
• ACKNOWLEDGED: The project team is aware of this finding. Fixes for this finding are planned. This finding does not affect the overall security of the project.
• NO ISSUE: The finding does not affect the overall security of the project and does not violate its operational logic.

Good, now let’s delve into what interesting findings we have discovered in our reports. Here are some intriguing figures.

In total, for 2023, we audited no less than 46,764 lines of code in more than 15 audits, where we identified a total of 529 problems. Oxorio audits only large companies with strong development teams, which allows you to somewhat replicate the conclusions made in the study to all other protocols. We also pay great attention to assessing timelines and do not conduct audits for less than 3 weeks, with the average duration of our audits being approximately 4 weeks.

First, let’s look at the diagram that shows the average concentration of vulnerabilities of each class per 1,000 lines of code:

As you can see, there are about 11 vulnerabilities per 1,000 lines of code on average, of which 3.57 (CRITICAL + MAJOR) are really serious issues, the failure to resolve which entails huge security risks.

This indicator can be useful for developers as an average industry benchmark during the internal security evaluation process. You can calculate this indicator for any protocol when familiarizing yourself with the results of its audit, and it will help understand how crucial the audit will be for future updates to your codebase.

Now let’s look at how vulnerabilities are distributed by Severity Level:

The high number of WARNING class vulnerabilities indicates that, despite the experience of development teams, many potential problems remain unaddressed in the early stages of design and development. This points to the need for stricter quality control and attention to fine details, which cannot be ensured without the involvement of auditors who focus on this purposefully.

Speaking of the relative rarity of CRITICAL vulnerabilities compared to MAJOR, it suggests that development teams effectively manage risks at the highest level, focusing efforts on preventing the most serious threats that could have catastrophic consequences. However, this may also indicate a tendency to underestimate less critical but still significant problems, leading to an increase in the number of MAJOR vulnerabilities. In turn, MAJOR vulnerabilities are dangerous because they often escalate to CRITICAL vulnerabilities.

As seen from the pie chart, nearly a third of the vulnerabilities (CRITICAL + MAJOR = 31.6%) are truly serious for protocol security and were missed even by very experienced developers. In one of the following sections, we’ll discuss in more detail what conclusions can be drawn from this and what can be corrected.

Now let’s look at the distribution of vulnerabilities by their types. But before that, we want to explain to you the methodology of this distribution. Certainly, the analysis looks somewhat generalized when it presents a limited number of types of vulnerabilities, as in practice, there are combined and overly specific problems that are difficult to classify unambiguously. However, we manually analyzed each vulnerability and identified 6 of the most common and frequently occurring types:

We have studied all vulnerabilities according to the methodology described above, and here is a diagram reflecting the results - let’s look and analyze.

As you can see from the graph, almost 37% of the findings are recommendations for optimization and simplification of code maintenance. This suggests that developers often either lack the time for thorough code polishing or simply don’t always see advanced ways to improve the quality of the code.

Even more interestingly, over a quarter (28.5%🤯) of vulnerabilities are embedded in the architecture of protocols at the logic development stage! More precisely, during the design phase, the existence of these attack vectors is simply not taken into account, resulting in significant changes being made post-audit, or even entirely rebuilding some application components.

What’s even more peculiar is that 24.5% of vulnerabilities still occur due to a lack of validation. It would seem that DeFi has vast experience with hacks due to this problem, and experienced developers (as in our case) have learned to closely monitor all function inputs. However, as practice shows, the number of non-obvious ways to bypass most checks is much greater than the resources development teams can physically allocate to finding and fixing them. Here it’s hardly possible to do without external help.

Alright, we’ve dealt with the vulnerabilities that auditors find. But how do clients process the findings from audit reports? To answer this question, let’s analyze the status of vulnerabilities that clients provide as feedback after reviewing the report. For convenience, we’ve also merged the data into a pie chart.

As the visualization shows, only a third of all problems are corrected. This alarmingly low value. We call this a serious problem for the reason that uncorrected findings, which the project’s developers did not pay enough attention to and just published in the final report, can be studied by hackers and then developed into MAJOR or CRITICAL vulnerabilities. Such a situation prompts auditors (including us) to interact more closely with their clients and inform them of potential consequences.

So, now we have all the data at hand. It’s time to articulate important theses and conclusions that we’ve drawn from this analysis. That’s what we’ll do in the next section!

Lessons to be learned

1. Despite the industry being nearly a decade old, we still observe persistent basic errors, thoroughly documented in numerous reports. This indicates that audits are still very important, and relying solely on experienced developers is like setting a time bomb under your project. We recommend founders and project managers not to devalue the importance of audits until there are substantial proofs that developers can independently manage security risks (which is very unlikely to happen in the next 3-4 years).
2. The statistics on Issue Status show that not all projects can pass a reaudit on the first attempt. Some of them, following the audit and even the reaudit, decide to halt deployment and send their code for partial or complete rework, as often addressing vulnerabilities requires significant reconsideration of the protocol’s architecture.
3. Also, the statistics on the rectification of vulnerabilities tell us that only 1/3 of vulnerabilities are fixed. We’ve already discussed how this increases risks for the protocol, but the key lesson here is that some companies are not sufficiently aware of the importance of the reaudit procedure, and some simply do it “for show,” which certainly needs to be rectified - by paying more attention to correcting the vulnerabilities found by auditors.
4. As the situation with infrastructure attacks has shown, research in the field of protocol security requires using a combination of research types: classic smart contract audits are increasingly needing to be reinforced with formal verification, penetration testing, and fuzz testing. Hackers are becoming smarter, their hacking approaches more sophisticated, and in response, security practices must also evolve. A security review using a combination of the aforementioned methodologies provides truly reliable protection against hacks of all sorts.

Oxorio’s contribution to strengthening security

1. Throughout the year, our team of auditors has made significant strides in mastering the audits and development of ZK solutions. After intensive theoretical training, one part of our team received several grants for the development of ZKPs, while another focused on creating a universal methodology for auditing ZK and studying practices for dealing with ZK vulnerabilities. We see great potential in this direction, and we are ready to assist you with ZK tasks of any complexity - we have top specialists, advanced practical experience, and an understanding of how vulnerabilities in ZK differ from or resemble blockchain vulnerabilities.
2. This year, we successfully completed a comprehensive audit of both the on-chain and off-chain parts of Lido V2, which significantly strengthened our expertise in LSD protocols. We thoroughly studied all the theoretical aspects and nuances of liquid staking, managed to find a large number of vulnerabilities, and proposed several innovative solutions for resolving architectural contradictions. In 2024, we plan to participate in at least several LSD protocol audits to become leaders in the field of liquid staking security.
3. Throughout 2023, we worked on creating internal practices for the continuous professional development of our audit team, researched many new attack vectors, and published over 15 studies and public reports on the most pressing topics of the year.

Looking to the future - challenges ahead

Today, it’s impossible to say definitively what kinds of hacks 2024 will bring. However, it becomes quite evident that with each passing year, hackers are becoming more experienced, and the architecture of blockchain solutions is becoming more branched and complex. Understanding the mechanisms of these applications requires increasingly in-depth analysis.

At the same time, this trend leads to an increase in the number of uncovered cases of vulnerabilities that allow hackers to conduct more complex, intricate attacks.

Based on this, we anticipate that hacks will become less obvious, combining attacks on both off-chain and on-chain components. And the protocols that are unable to prepare for this face the risk of experiencing types of breaches that the industry has not seen before.

Conclusion

We have tried to reflect our technical view on the most important events and trends that took place in 2023 in the field of Blockchain Security. We highlighted 5 main narratives, analyzed and visualized the results of all the audits we conducted, summed up our company’s results, and looked into the future to understand what 2024 holds for us.

We just want to reemphasize that our team is closely interacting with most of the cutting-edge technologies and significant industry updates, and we are ready to assist you with any security task that your protocols face.

We wish you a Happy New Year and a Merry Christmas and hope that from our thoughts you were able to catch important insights that will help us make blockchain development safer together.