a
I am a highly motivated and organised professional with more than ten years of experience as a Database Specialist and Architect or designer.
Bring Me a Coffee - NZ$ 5

Skill No 5 – Machine Learning and AI

Select AI – Natural Language to SQL Generation on Autonomous Database

Autonomous Database Select AI, which enables you to easily retrieve data using natural language. Combining generative AI with Oracle SQL allows you to express your desired outcome and let the database generate the appropriate SQL query for your schema. While some language models may excel at generating SQL, executing it on your database can be a challenge. Select AI streamlines the process by generating SQL tailored to your specific database.

Large language models (LLMs) powered by AI are trained on vast amounts of text data and can understand the nuances and intended meaning of most natural language inquiries. By using natural language to create SQL, the time taken to generate queries is reduced, query construction is simplified, and the need for specialized SQL expertise is minimized or eliminated. Obtaining information from your database that involves creating queries with multiple joins, nested subqueries, and other SQL structures becomes easier and faster when using plain language.

By learning effective SQL query patterns from curated training data, LLMs can produce more efficient queries – enabling them to perform better. As part of Oracle Autonomous Database, Select AI inherits all security and authentication features of the database.

By virtue of being integrated with Oracle SQL, this capability is available through any SQL IDE, SQL Developer Web, Oracle Application Express (APEX), and Oracle Machine Learning Notebooks. Any application that invokes SQL and has an AI provider account also has access to Select AI.

Large Language are Insufficient

Certain LLMs excel at generating SQL code and are capable of accurately interpreting the user’s intentions. These LLMs can even generate table and field names that fit within the syntax of a valid SQL query. However, it’s important to note that this query will only work if your database has pre-existing tables with columns. Essentially, the LLM is able to imagine the necessary tables and fields in order to generate the SQL code.

To generate a runnable query, we must first tie it to a certain schema, which is where Select AI comes in. By creating a profile, you can use your local schema by default, but you can also indicate which schema(s) and table(s) you wish to be considered when producing queries.

The initial natural language query is enhanced with metadata from schema(s) found in the user’s profile, as shown below. Feeding information to the LLM allows it to generate a SQL query that can be executed against your database and, hopefully, offer the results you’re searching for.

DBMS_CLOUD_AI Package

Enabling this feature is a new package, DBMS_CLOUD_AI, in Autonomous Database that enables the use of LLMs for generating SQL from natural language prompts. The package provides access to user specified LLMs along with knowledge of the user’s accessible database metadata. This enables producing runnable SQL queries applicable to that schema. The DBMS_CLOUD_AI package currently integrates with AI providers such as OpenAI and Cohere, with others planned to follow. To use this feature, you need an account with the AI provider and must supply your API credentials to Autonomous Database.

Getting started

To get started with Select AI, sign into your Autonomous Database instance with administrator privileges and add your user (here MY_USER) to the ACL list and grant access to the DBMS_CLOUD_AI package:

Copy Code
BEGIN DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE( host => ‘api.openai.com’, ace => xs$ace_type(privilege_list => xs$name_list(‘http’), principal_name => ‘MY_USER’, principal_type => xs_acl.ptype_db) ); END; grant execute on DBMS_CLOUD_AI to MY_USER;

Create a database credential to your AI provider account. In the case of OpenAI, the password is the uniquely generated token for API usage.

Copy Code
BEGIN DBMS_CLOUD.DROP_CREDENTIAL ( credential_name => ‘OPENAI_CRED’); DBMS_CLOUD.CREATE_CREDENTIAL( credential_name => ‘OPENAI_CRED’, username => ‘OPENAI’, password => ‘xyz’ ); END;

Create your DBMS_CLOUD_AI profile. Here, we list the schema(s) we want to be considered (e.g., SH) and, optionally, any tables (e.g., customers, sales, etc.).

Copy Code
BEGIN DBMS_CLOUD_AI.drop_profile(profile_name => ‘OPENAI’); DBMS_CLOUD_AI.create_profile( profile_name => ‘OPENAI’, attributes => ‘{“provider”: “openai”, “credential_name”: “OPENAI_CRED”, “object_list”: [{“owner”: “SH”, “name”: “customers”}, {“owner”: “SH”, “name”: “sales”}, {“owner”: “SH”, “name”: “products”}, {“owner”: “SH”, “name”: “countries”}] }’); END;

Set your DBMS_CLOUD_AI profile, which must be done in each database session.

Copy Code
BEGIN DBMS_CLOUD_AI.SET_PROFILE( profile_name => ‘OPENAI’ ); END;

Very Soon I will put more information to complement this new feature.

AI and Data Management: A Powerful Partnership

Artificial intelligence (AI) is transforming every industry and every aspect of our lives. From healthcare to education, from entertainment to finance, AI is enabling new possibilities and creating new value. But AI is not a magic wand that can solve any problem without any effort. AI depends on data – lots of data – to learn, improve, and deliver accurate and reliable outcomes. And data needs management – effective management – to ensure its quality, accessibility, security, and integration.

What exactly does a Data Management Engineer do?

Data management is the process of collecting, organizing, storing, processing, analyzing, and sharing data in a way that supports business goals and complies with regulations. Data management involves various tasks such as data classification, cataloguing, quality control, security enforcement, and data integration. These tasks are often labour-intensive and error-prone when done manually or with traditional tools.

How AI can help to do our job more efficiently?

This is where AI can help. AI can automate and simplify tasks related to data management across discovery, integration, cleansing, governance, and mastering. AI can improve data understanding and identify privacy and quality anomalies. AI can also enhance data security by detecting threats and enforcing policies. AI can enable data integration by matching records across sources and resolving conflicts.
By using AI for data management, Database Specialist can benefit from:
  1. Improved productivity: AI can reduce the time and effort required for data management tasks by automating repetitive or complex operations.
  2. Increased accuracy: AI can improve the quality and consistency of data by identifying errors or inconsistencies and correcting them.
  3. Enhanced scalability: AI can handle large volumes of data from various sources without compromising performance or reliability.
  4. Greater agility: AI can adapt to changing business needs or regulatory requirements by learning from feedback or new information.
  5. Higher value: AI can unlock the potential of data by providing insights or recommendations that support decision-making or innovation.
Some examples of how AI can be useful for data management are:
  • Classification: AI can extract relevant information from documents, images, videos, or other media using natural language processing (NLP), computer vision (CV), or speech recognition techniques.
  • Cataloguing: AI can help locate data by indexing it based on metadata or content analysis using NLP or CV techniques.
  • Quality: AI can reduce errors in the data by validating it against rules or standards using NLP or machine learning (ML) techniques.
  • Security: AI can keep data safe from unauthorized access or misuse by applying encryption or masking techniques using ML techniques.
  • Integration: AI can help merge data from different sources by matching records based on similarity measures using ML techniques.
  • Data Cleansing: AI can automatically identify and correct errors in data by using machine learning algorithms. For example, AI can identify and remove duplicate data, identify outliers, and standardize data.
  • Data Governance: AI can help enforce data governance policies by identifying and flagging any inconsistencies or violations in the data. AI can also provide recommendations for policies based on the analysis of the data.
  • Data Discovery: AI can help discover hidden patterns and insights in large datasets that are difficult for humans to uncover. For example, AI can identify trends and correlations between different data sets and provide recommendations for further analysis.
  • Data Mastering: AI can help improve data accuracy and completeness by merging, standardizing, and de-duplicating data from different sources. This can help reduce data inconsistencies and improve data quality.
AI and data management are a powerful partnership that can enable organizations to leverage their most valuable asset – their data – in a more efficient and effective way. By combining human expertise with machine intelligence, organizations can achieve better outcomes faster while reducing costs and risks. If you want to learn more about how AI can help you with your data management challenges, please contact me, and we will be happy to assist you.
Regards;