By Gourab Mitra, Jozsef Patvarczki, Tony Petukhov, Mike Waas
Every database user has been there: your SQL statement doesn’t work. At first, it’s unclear if it is a matter of wrong syntax, a limitation of the language support of the database, or simply a bug.
To make your case with the database vendor—be it a support ticket or a feature request— you want to provide as much detail as possible. Specifically, the vendor will ask for a repro of the problem. That seems fair. However, spending countless hours creating a concise repro is the last thing you need right now.
As a result, many issue reports are never filed. This lack of critical feedback hurts both users and vendors. The opportunity to improve end-user productivity is squandered—the chance to squash a bug when it first surfaces is missed.
Ultimately, vendors can’t help but play by the adage that “the squeaky wheel gets the oil.” Whoever has the most resources to make their case or is the most vocal will get their attention. It stands to reason that most vendors build new features based on emotional feedback rather than hard data.
We wanted to avoid this trap when building OpenDB. Because of the nature of OpenDB, a drop-in replacement of Oracle, we want to know how our customers are trying to use features that are yet to be supported. Eliminating the hurdles of an arduous bug filing process would be essential to quickly building the most complete coverage.
OpenDB — a new kind of database
First, let’s talk about OpenDB. OpenDB is a unique kind of database. In a nutshell, OpenDB enables you to run Oracle workloads on PostgreSQL without modifying your Oracle SQL. We’re not only talking about schema and procedures in the database. We’re talking about actual application code: Unlike a conventional database migration, where you must run your code through code converters, OpenDB lets you use your app as is—just replace your driver. As you appreciate, OpenDB is a unique kind of database with a strong emphasis on feature completeness.
Users of OpenDB run workloads that work as-is on Oracle on our platform. If there’s a feature that OpenDB doesn’t yet support, they will see either a syntax error or a custom error message that indicates this feature is not yet supported.
For the ongoing development of OpenDB, it’s critically important for us to understand what features are missing. How did the user try to use the feature? What was the context in which they tried it? What objects did they reference, and what data types were involved?
However, asking a customer to file feature requests is rarely helpful. Instead, it only netted us high-level descriptions. Getting to a more nuanced description and even specific corner cases, relying on the customer’s best-effort description, only sometimes resulted in actionable feedback for our product team.
In short, we needed a mechanism that makes it easy for customers to share their experience and their feature requests without burdening them with additional work.
Building an automatic feature request mechanism
The idea behind Feature Request Feedback (FRF) is as simple as it is striking. Instead of asking a customer to formulate their feature request, simply capture the context in which they tried to use that feature and turn it into a concise repro.
The critical insight underlying FRF is that at the time when the unsupported feature is encountered, the offending statement and the entire context, including schema information, are known to the system. Hence, packing up the information and sharing it with Datometry is a natural next step: At the time of the error, i.e., attempt to use a feature that isn’t supported in a specific context, FRF collects all elements needed for a repro, including schema and system information, and collates them into a self-contained report.
The lexical analysis of the offending statement is likely already complete. Most, if not all, object references may have been resolved already against the existing metadata. Since we added complete syntactic analysis for Oracle to OpenDB, all Oracle syntax is accepted even if it errors out later.
FRF is much like error reports that most users are familiar with from their laptops. The big difference is, however, that we can reconstruct a concise repro that captures the last steps of what the user tried to accomplish. FRFs do not need to pop up a dialog asking the user to describe what they were doing—we can reconstruct that information conclusively.
FRF is simple, fully automatic, and conclusive.
Early experience with FRF
We released an OpenDB version with FRF in Q2’24 via AWS Marketplace. Within just a few months, our automation collected over 8,000 FRFs. To get a sense of the space, we collected FRFs upon any error the system raised. This also includes a fair number of syntax errors triggered by bad queries that do not succeed on Oracle either.
By analyzing these reports, an interesting picture emerged. First off, many queries, especially very simple system queries, were duplicates! That’s often because they are standing idioms or emitted by standard frameworks. Because OpenDB is agnostic of the applications that submit the queries, FRF is impartial. Unlike a conventional product management process, FRF is not unconsciously biased toward specific applications or customers.
By deduping on query text, the number of FRF shrank considerably. Further deduping reports by error message eliminates those that point to the same problem but are triggered by different queries. Collating these into one category provides us with a more complete picture of how exactly a given feature is being used. One might even think of comprehensive test case generation.
Our initial experience has been overwhelmingly positive. Among the things that work well is, first and foremost, the identification of features that are not yet supported. So, no surprise here. Another interesting category is the analysis of internal errors. The FRFs, in this case, let us pinpoint the root cause much faster than with any conventional method.
Going forward, we also see room for improvement. False positives may arise if a trigger event results from a previously encountered issue. For example, if there’s a missing feature in the definition of a view, all subsequent queries that try to use the view will fail as the object couldn’t be created earlier. We refer to this error category as “object not found” issues or ONF for short.
ONF reports can be ignored easily from further processing. However, ignoring them in the analysis is a double-edged sword. ONFs mean only a part of the query has been analyzed; thus, additional issues might be missed. For now, the false positives are but a minor nuisance. In upcoming releases, we will probably exclude ONFs from the FRF process.
Stepping it up
Our experience with FRF has been overwhelmingly positive and, in many respects, quite an eye-opener! We’re grateful to our users and elated that we can give them an active role in product development.
OpenDB v2.5 is now available on AWS. We’re proud to report that most of the issues detected with FRF have been addressed. It has been quite a unique experience to retrieve this level of customer feedback without the familiar, labor-intensive—and often inadvertently biased—process of product management.
So, go take it out for a spin! Help us widen the aperture for further feature requests and improve the usability of OpenDB—by simply using OpenDB. Keep in mind that the marketplace version is limited in functionality compared to the enterprise version, so feel free to reach out if you have an enterprise-grade migration project. Happy migrating and happy querying!
About Datometry
Datometry is the leader in database system virtualization. With Datometry Hyper-Q, enterprises can run their existing applications directly on next-generation databases without needing costly and risk-laden database migrations. Datometry counts leading Fortune 500 and Global 2000 enterprises worldwide among their customers. For more information, visit www.datometry.com
