Migrate golang install linux

Saved searches

Use saved searches to filter your results more quickly

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Database migrations. CLI and Golang library.

License

golang-migrate/migrate

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Update spanner to fix security issue

Git stats

Files

Failed to load latest commit information.

README.md

Database migrations written in Go. Use as CLI or import as library.

• Migrate reads migrations from sources and applies them in correct order to a database.
• Drivers are «dumb», migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
• Database drivers don’t assume things or try to correct user input. When in doubt, fail.

Database drivers run migrations. Add a new database?

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: ! , # , $ , % , & , ‘ , ( , ) , * , + , , , / , : , ; , = , ? , @ , [ , ]

It’s easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

$ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))' String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D $ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")' String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D $

Source drivers read migrations from local or remote sources. Add a new source?

• Filesystem — read from filesystem
• io/fs — read from a Go io/fs
• Go-Bindata — read from embedded binary data (jteeuwen/go-bindata)
• pkger — read from embedded binary data (markbates/pkger)
• GitHub — read from remote GitHub repositories
• GitHub Enterprise — read from remote GitHub Enterprise repositories
• Bitbucket — read from remote Bitbucket repositories
• Gitlab — read from remote Gitlab repositories
• AWS S3 — read from Amazon Web Services S3
• Google Cloud Storage — read from Google Cloud Platform Storage

• Simple wrapper around this library.
• Handles ctrl+c (SIGINT) gracefully.
• No config search paths, no config files, no magic ENV var injections.

$ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2

$ docker run -v >:/migrations --network host migrate/migrate -path=/migrations/ -database postgres://localhost:5432/database up 2

• API is stable and frozen for this release (v3 & v4).
• Uses Go modules to manage dependencies.
• To help prevent database corruptions, it supports graceful stops via GracefulStop chan bool .
• Bring your own logger.
• Uses io.Reader streams internally for low memory overhead.
• Thread-safe and no goroutine leaks.

import ( "github.com/golang-migrate/migrate/v4" _ "github.com/golang-migrate/migrate/v4/database/postgres" _ "github.com/golang-migrate/migrate/v4/source/github" ) func main() < m, err := migrate.New( "github://mattes:personal-access-token@mattes/migrate_test", "postgres://localhost:5432/database?sslmode=enable") m.Steps(2) >

Want to use an existing database client?

import ( "database/sql" _ "github.com/lib/pq" "github.com/golang-migrate/migrate/v4" "github.com/golang-migrate/migrate/v4/database/postgres" _ "github.com/golang-migrate/migrate/v4/source/file" ) func main() < db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable") driver, err := postgres.WithInstance(db, &postgres.Config<>) m, err := migrate.NewWithDatabaseInstance( "file:///migrations", "postgres", driver) m.Up() // or m.Step(2) if you want to explicitly set the number of migrations to run >

Источник

How to Perform Database Migrations using Go Migrate

Rwitesh Bera

Since its introduction, the programming language Go (also known as Golang) has become increasingly popular. It is known for its simplicity and efficient performance, similar to that of a lower-level language such as C++.

While working with a database, schema migration is one of the most important tasks that developers do throughout the project lifecycle. In this article, I will explain what database migration is and how to manage it using go-migrate.

What is a Database Migration?

A database migration, also known as a schema migration, is a set of changes to be made to a structure of objects within a relational database.

It is a way to manage and implement incremental changes to the structure of data in a controlled, programmatic manner. These changes are often reversible, meaning they can be undone or rolled back if required.

The process of migration helps to change the database schema from its current state to a new desired state, whether it involves adding tables and columns, removing elements, splitting fields, or changing types and constraints.

By managing these changes in a programmatic way, it becomes easier to maintain consistency and accuracy in the database, as well as keep track of the history of modifications made to it.

Setup and Installation

migrate is a CLI tool that you can use to run migrations. You can easily install it on various operating systems such as Linux, Mac and Windows by using package managers like curl, brew, and scoop, respectively.

For more information on how to install and use the tool, you can refer to the official documentation.

To install the migrate CLI tool using scoop on Windows, you can follow these steps:

To install the migrate CLI tool using curl on Linux, you can follow these steps:

$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey| apt-key add - $ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list $ apt-get update $ apt-get install -y migrate

To install the migrate CLI tool using on Mac, you can follow these steps:

$ brew install golang-migrate

How to Create a New Migration

Create a directory like database/migration to store all the migration files.

Next, create migration files using the following command:

$ migrate create -ext sql -dir database/migration/ -seq init_mg

You use -seq to generate a sequential version and init_mg is the name of the migration.

A migration typically consists of two distinct files, one for moving the database to a new state (referred to as «up») and another for reverting the changes made to the previous state (referred to as «down»).

The «up» file is used to implement the desired changes to the database, while the «down» file is used to undo those changes and return the database to its previous state.

The format of those files for SQL are:

When you create migration files, they will be empty by default. To implement the changes you want, you will need to fill them with the appropriate SQL queries.

How to Run Migration Up

In order to execute the SQL statements in the migration files, migrate requires a valid connection to a Postgres database.

To accomplish this, you will need to provide a connection string in the proper format.

$ migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose up

Now in your Postgres shell, you can check newly created tables by using the following commands:

\d+ \d+ table_name DESCRIBE TABLE

How to Rollback Migrations

If you want to revert back the migration, you can do that using the following down tag:

$ migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose down

It will delete the email column from both tables as mentioned in the 000002_init_mg.up.sql file.

Now, let’s check the database and see if email has been deleted or not:

How to Resolve Migration Errors

If a migration contains an error and is executed, migrate will prevent any further migrations from being run on the same database.

An error message like Dirty database version 1. Fix and force version will be displayed, even after the error in the migration is fixed. This indicates that the database is «dirty» and needs to be investigated.

It is necessary to determine if the migration was applied partially or not at all. Once you’ve determined this, the database version should be forced to reflect its true state using the force command.

$ migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" force

How to Add Commands in a Makefile

migration_up: migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose up migration_down: migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose down migration_fix: migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" force VERSION

Now, you can run $ make migration_up for ‘up’, $ make migration_down for ‘down’, and $ make migration_fix to fix the migration issue.

Before running the makefile, ensure that the correct version number is included in the migration_fix command.

Conclusion

Migration systems typically generate files that can be shared across developers and multiple teams. They can also be applied to multiple databases and maintained in version control.

Keeping a record of changes to the database makes it possible to track the history of modifications made to it. This way, the database schema and the application’s understanding of that structure can evolve together.

That concludes our discussion on database migration. I hope you found the information to be useful and informative.

If you enjoyed reading this article, please consider sharing it with your colleagues and friends on social media. Additionally, please follow me on Twitter for more updates on technology and coding. Thank you for reading!

Источник