Generative AI function GENERATE_TEXT in BigQuery

Since I started working with the Google Platform, Google has not stopped surprising me with its BigQuery (BQ) features and development.

A real “wow” moment for me happened four years ago.

I remember it like it was yesterday, and I was sitting in the front row at the Big Data London 2019 conference. Little did I know back then about the possibility of creating machine learning models using only BQ functions, or, better said, what BQ Machine Learning (BQML) is.

At least until the conference session, where the Google colleague presented how you can create classification, clustering, and time-series forecasting models by simply using Google’s SQL.

The first thought that went through my mind back then was “You must be joking”.

The second thought in my head was, “Does this mean that everyone who knows only SQL will be able to create machine learning models?”

As you can assume, the answer is “yes” if you are using BigQuery as your data warehouse.

Now, after using the BQML functions for a while, the correct answer to the question listed above is “maybe.”

This means that even though the CREATE MODEL syntax is written in SQL, knowledge of machine learning modelling and statistics is still needed.

In other words, you still need to understand the math behind the available models for different types of machine learning use cases (supervised/unsupervised), conduct feature engineering, hyperparameter tuning and model evaluation tasks.

Fast forward to the year 2023, and BigQuery is further amazing me with its new features.

This time, we are talking about the new generative AI BigQuery machine learning functions.

With these new functions, data engineers and analysts are able to perform generative natural language tasks on textual data stored in BQ tables with a few query lines.

Hence, the goal of this blog post is to showcase the new analytical advances of BQ in generative AI, with a focus on one function—the…