Hi and welcome back! In the previous lectures, we’ve learned how to write golang codes
to perform CRUD operations on each individual table of the simple bank
database. But in a real world application, we often have to perform a transaction that combines some operations from several
tables. Today we will learn a how to to implement
it in Golang. Before we jump into coding, Let’s talk a bit about transaction. What is a database transaction? Well, basically, It’s a single unit of work That’s often made up of multiple database
operations. For example, In our simple bank, we want to transfer 10 USD from account 1
to account 2. This transaction comprises 5 operations: First, we create a transfer record with amount
equals to 10. Second, we create an account entry record
for account 1 with amount equals to -10, since money is moving out of this account. Third, we create another account entry record
for account 2, but with amount equals to 10, because money is moving in to this account. Then we update the balance of account 1 by
subtracting 10 from it. And finally we update the balance of account
2 by adding 10 to it. This is the transaction that we’re going
to implement in this video. We will come to that in a moment. But why do we need to use database transaction? There are 2 main reasons: First, we want our unit of work to be reliable
and consistent, even in case of system failure. And second, we want to provide isolation between
programs that access the database concurrently. In order to achieve these 2 goals, a database transaction must satisfy the ACID
properties. Where A is Atomicity, which means either all operations of the transaction
complete successfully, or the whole transaction fails, and everything
is rolled back, and the database is unchanged. C is Consistency, which means the database state should remains
valid after the transaction is executed, More precisely, all data written to the database
must be valid according to predefined rules, including constraints,
cascades, and triggers. I is Isolation, meaning all transactions that run concurrently
should not affect each other. There are several levels of isolation that defines when the changes made by 1 transaction
can be visible to others. We will learn more about it in another lecture. The last property is D, which stands for Durability. It basically means that all data written by
a successful transaction must stay in a persistent storage, and cannot be lost, even in case of system
failure. OK, so how to run a SQL database transaction? It’s pretty simple. We start a transaction with the BEGIN statement. Then we write a series of normal SQL queries
(or operations). If all of them are successful, We COMMIT the transaction to make it permanent, The database will be changed to a new state. Otherwise, if any query fails, we ROLLBACK the transaction, thus all changes made by previous queries
of the transaction will be gone, and the database stays the same as it was
before the transaction. Alright, Now we has some basic understanding about
database transaction. Let’s learn how to implement it in Golang. I’m gonna create a new file store.go inside
the db/sqlc folder. In this file, let’s define a new Store struct. This store will provide all functions to run database queries individually, as well as their combination within a transaction. For individual queries, we already have the Queries struct generated
by sqlc that we have learn in previous lectures. However, each query only do 1 operation on
1 specific table. So Queries struct doesn’t support transaction. That’s why we have to extend its functionality by embedding it inside the Store struct like
this. This is called a composition, and it is the preferred way to extend struct
functionality in Golang instead of inheritance. By embedding Queries inside Store, All individual query functions provided by
Queries will be available to Store. And we can support transaction by adding more
functions to that new struct. In order to do so, we need the Store to have
a sql.DB object. because it is required to create a new db
transaction. OK, now let’s add a function to create a
new Store object. It will takes a sql.DB as input, and returns
a Store. Inside, we just build a new Store object and
return it. Where db is the input sql.DB, and Queries is created by calling the New
function with that db object. This New function was generated by sqlc as we’ve already known in the previous lectures. It creates and returns a Queries object. Next, we will add a function to the Store to execute a generic database transaction. The idea is simple, It takes a context and a callback function
as input, Then it will start a new db transaction, Create a new Queries object with that transaction, And call the callback function with the created
Queries, And finally commit or rollback the transaction based on the error returned by that function. Alright, let’s implement this. First, to start a new transaction, we call store.db.BeginTx() Pass in the context, And optionally a TxOption. This option allows us to set a custom Isolation
Level for this transaction. If we don’t set it explicitly, then the default isolation level of the database
server will be used, which is read-committed in case of Postgres. We will learn more about this in another lecture, For now, let’s just pass nil here to use
the default value. The BeginTx() function returns a transaction
object or an error. If error is not nil, we just return it immediately. Otherwise, we call New() function with the
created transaction And get back a new Queries object. This is the same New() function that we used
in the NewStore() function, The only difference is, instead of passing
in a sql.DB, We’re now passing in a sql.Tx object. This works because the New() function accepts
a DBTX interface as we’ve seen in the previous lecture. OK, now we have the queries that runs within
transaction, We can call the input function with that queries, And get back an error. If the error is not nil, Then we need to rollback the transaction We do that by calling tx.Rollback(). It also returns a rollback error. If the rollback error is also not nil, then we have to report 2 errors. So we should combine them into 1 single error
before returning. To do that, we use fmt.Errorf command, First we add the transaction error, Then the rollback error. In case the rollback is successful, we just return the original transaction error. Finally, If all operations in the transaction are successful, We simply commit the transaction with tx.Commit(), and return its error to the caller. And we’re done with the execTx() function. Note that this function is unexported (it starts
with a lowercase letter e), because we don’t want external package to
call it directly. Instead, we will provide an exported function
for each specific transaction. Now let’s go a head and add a new TransferTx()
function to perform the money transfer transaction
example that we discussed at the beginning of the
video. To recall, it will create a new transfer record, add new account entries, and update accounts’ balance within a single database transaction. The input of this function will be a context And an argument object of type TransferTxParams And it will return a TransferTxResult object
or an error. Now let’s define the TransferTxParams. This struct contains all necessary input parameters
to transfer money between 2 accounts, FromAccountID is the ID of the account where
money will be sent from. ToAccountID is the ID of the account where
money will be sent to. And the last field is the amount of money
to be sent. The TransferTxResult struct contains the result
of the transfer transaction. It will have 5 fields: The created transfer record. The from account after its balance is updated. The to account after its its balance is updated. The entry of the from account which records
that money is moving out. And the entry of the to account which records
that money is moving in. Alright, now we can implement the transfer
transaction. First we create an empty result. Then we call the store.execTx() function that
we’ve written before to create and run a new database transaction. Pass in the context and the callback function. For now let’s just return nil. Finally we return the result and the error
of the execTx() call. Now let’s come back to implement this callback
function. Basically, we can use the queries object to call any individual CRUD function that
it provides. Keep in mind that this queries object is created
from 1 single database transaction, So all of its provided methods that we call
will be run within that transaction. Alright, let’s create the transfer record
by calling q.CreateTransfer() Pass in the input context, and a CreateTransferParams Where FromAccountID is arg.FromAccountID ToAccountID is arg.ToAccountID And Amount is arg.Amount The output transfer will be saved to result.Transfer And we also have to declare this error variable. Then here we check If error is not nil, just return it right
away. Now here you can see that we’re accessing the result variable of the
outer function from inside this callback function. similar for the arg variable. This makes the callback function become a
closure. Since Go lacks support for generics type, Closure is often used when we want to get
the result from a callback function, because the callback function itself doesn’t
know the exact type of the result it should return. OK so the 1st step to create a transfer record
is done. Next step is to add 2 account entries, 1 for the from account, and 1 for the to account. So result.FromEntry, error equals to q.CreateAccountEntry() We pass in the context and a CreateAccountEntryParams, where AccountID is arg.FromAccountID, and Amount is -arg.Amount because money is
moving out of this account. And just like before, If error is not nil, we just return it so that the transaction will be rolled back. We do similar thing to create an account entry
for the to account. This time, it is result.ToEntry the AccountID is arg.ToAccountID, And the amount is just arg.Amount since money
is moving in to this account. And we’re done with the account entries
creation. The last step to update account balance will
be more complicated because it involves locking and preventing
potential deadlock, So I think it’s worth a separate lecture
to talk about it in details. For now, let’s add a TODO comment here, and we will come back to implement it in the
next video. Now let’s say our TransferTransaction is
done with 1 transfer record and 2 account entries
are created. We have to test it to make sure that it’s
working as expected. I’m gonna create a new store_test.go file. It’s in the same db package as our store.go Then let’s define a new unit test for the
TransferTx() function. First we need to create a new Store object. The NewStore() function requires a sql.DB
object, If you still remember, in the previous lecture, we have already created a sql.DB object in the main_test.go file with this sql.Open() call. So in order to reuse it, here instead of assigning the result to the
connection variable, we will declare a new global variable: testDB And store the result of the sql.Open() command
in it. We should remove this colon because testDB
is not a new variable. And therefore, we must declare the error up
here to make it an existed variable. Finally we have to change this connection
variable to testDB when creating the testQueries object. OK, now we can come back to our unit test and pass the testDB into this function to
create a new Store. Next, we will create 2 random accounts using
the function we wrote in the previous lecture. We will send money from account 1 to account
2. From my experience, writing database transaction is something
we must always be very careful with. It can be easy to write, but can also easily become a nightmare if we don’t handle the concurrency carefully. So the best way to make sure that our transaction
works well is to run it with several concurrent go routines. Let’s say I want to run n = 5 concurrent
transfer transactions And each of them will transfer an amount of
10 from account 1 to account 2. So I will use a simple for loop with n iterations And inside the loop, we use the go keyword to start a new routine. Make sure to have this bracket at the end
of the function to run it. Now inside the go routine, We call store.TransferTx() function with a
background context And a TransferTxParams object, where FromAccountID is account1.ID ToAccountID is account2.ID And Amount is 10 as we declared above. This function returns a result or an error. Now we cannot just use testify require to
check them right here because this function is running inside a
different go routine from the one that our TestTransferTx function
is running on, So there’s no guarantee that it will stop
the whole test if a condition is not satisfied. The correct way to verify the error and result
is to send them back to the main go routine that our test is running
on, and check them from there. To do that, we can use channels. Channel is designed to connect concurrent
Go routines, and allow them to safely share data with each
other without explicit locking. In our case, we need 1 channel to receive
the errors, And 1 other channel to receive the TransferTxResult. We use the make keyword to create the channel. Now, inside the go routine, we can send error to the errors channel using
this arrow operator, The channel should be on the left, and data to be sent should be on the right
of the arrow operator. Similarly, we send result to the results channel. Then, we will check these errors and results
from outside We simply run a for loop of n iterations, To receive the error from the channel, we use the same arrow operator, But this time, the channel is on the right
of the arrow, and the variable to store the received data
is on the left. We require no errors here, which means the received error should be nil. Likewise, we receive result from the results
channel Check that result is not an empty object. As result contains several objects inside, Let’s verify each of them. Start with the result.Transfer. We require this transfer object to be not
empty Then the from account ID field of transfer
should equal to account1.ID The to account ID field of transfer should
equal to account2.ID And transfer.Amount should equal to the input
amount. The ID field of transfer should not be zero
because it’s an auto-increment field And finally transfer.CreatedAt should not
be a zero value because we expect the database to fill in
the default value, which is the current timestamp. Now to be sure that a transfer record is really
created in the database, We should call store.GetTransfer() to find
the record with ID equals to transfer.ID. Here you can see that, because the Queries object is embedded inside
the Store, the GetTransfer() function of Queries is also
available to the Store. If the transfer really exists, this function should not return an error. So we require no error. OK, next we will check the account entries
of the result. First, the from entry. Just like before, We check that it should not be empty. The account ID should be account 1 And the amount of the entry should equal to
-amount because money is going out. Finally the ID and created at fields of the
entry should be not zero. We also try to get the account entry from
the database to make sure that it’s really got created. Checking the to entry is similar. So I will copy this block of code And change these variable and field names
to toEntry. This account ID should be account 2 instead
of account 1. And the amount should be positive instead
of negative because money is going in. In the end, we should get the toEntry record
instead. Now keep in mind that we should also check
the accounts’ balance as well, But since we haven’t implemented the part
to update accounts’ balance yet, Let’s add a TODO comment here for now, and we will complete it in the next lecture. Alright, now the test is ready Let’s try to run it. It passed! Excellent! Let’s run the whole package tests. All passed! The coverage is about 80%, Which is very good. And that wraps up today’s lecture about how to implement database transaction in Golang. I hope you enjoy it, You can try to implement the update account
balance yourself while waiting for the next video. Happy coding and see you in the next lecture!
