[bcdfc53] | 1 | <?xml version="1.0" encoding="ISO-8859-1"?>
|
---|
| 2 | <!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
|
---|
| 3 | "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
|
---|
| 4 | <!ENTITY % general-entities SYSTEM "../../general.ent">
|
---|
| 5 | %general-entities;
|
---|
| 6 | ]>
|
---|
| 7 |
|
---|
| 8 | <sect1 id="upgradedb" xreflabel="Upgrade Database">
|
---|
| 9 | <?dbhtml filename="upgradedb.html"?>
|
---|
| 10 |
|
---|
| 11 | <sect1info>
|
---|
| 12 | <date>$Date$</date>
|
---|
| 13 | </sect1info>
|
---|
| 14 |
|
---|
[7f6bf79b] | 15 | <title>Important notes about upgrading Database Server Software</title>
|
---|
| 16 |
|
---|
[d86ea4b] | 17 | <note><para>This section is about reinstalling database software
|
---|
| 18 | when an existing database is in use. It is not applicable for
|
---|
| 19 | initial installations or if there is no existing database for
|
---|
[b3ba4f9] | 20 | the package being updated, but users should read through it
|
---|
| 21 | to become aware of issues that can arise in the future.</para></note>
|
---|
[d86ea4b] | 22 |
|
---|
| 23 | <para>Lets start this chapter with a dramatic screenshot of an issue that
|
---|
| 24 | really happened. This issue will not occur if you are going to install the
|
---|
| 25 | software the first time:</para>
|
---|
[bcdfc53] | 26 |
|
---|
| 27 | <screen>$ sudo systemctl status postgresql
|
---|
| 28 | -- postgresql.service - PostgreSQL database server
|
---|
| 29 | Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
|
---|
| 30 | Active: failed (Result: exit-code) since Tue 2021-10-26 17:11:53 CDT; 2min 49s ago
|
---|
| 31 | Process: 17336 ExecStart=/usr/bin/pg_ctl -s -D ${PGROOT}/data start -w -t 120 (code=exited, status=1/FAILURE)
|
---|
| 32 | CPU: 7ms
|
---|
| 33 |
|
---|
| 34 | Oct 26 17:11:53 SVRNAME systemd[1]: Starting PostgreSQL database server...
|
---|
[d86ea4b] | 35 | Oct 26 17:11:53 SRVNAME postgres[17338]: 2021-10-26 17:11:53.420 CDT [17338] FATAL:
|
---|
| 36 | database files are incompatible with server
|
---|
| 37 | Oct 26 17:11:53 SRVNAME postgres[17338]: 2021-10-26 17:11:53.420 CDT [17338] DETAIL:
|
---|
| 38 | The data directory was initialized by PostgreSQL version 13,
|
---|
| 39 | which is not compatible with this version 14.0.
|
---|
[bcdfc53] | 40 | Oct 26 17:11:53 SRVNAME postgres[17336]: pg_ctl: could not start server
|
---|
| 41 | Oct 26 17:11:53 SRVNAME postgres[17336]: Examine the log output.
|
---|
| 42 | Oct 26 17:11:53 SRVNAME systemd[1]: postgresql.service: Control process exited, code=exited, status=1/FAILURE
|
---|
| 43 | Oct 26 17:11:53 SRVNAME systemd[1]: postgresql.service: Failed with result 'exit-code'.
|
---|
| 44 | Oct 26 17:11:53 SRVNAME systemd[1]: Failed to start PostgreSQL database server.</screen>
|
---|
| 45 |
|
---|
| 46 | <para>
|
---|
[d86ea4b] | 47 | To avoid situations like the one above finding your database server
|
---|
| 48 | software refusing to start, read the following thoughts about how to
|
---|
| 49 | upgrade a DBMS (Database Management System) prior to actually doing the
|
---|
[bcdfc53] | 50 | upgrade.
|
---|
| 51 | </para>
|
---|
[d86ea4b] | 52 |
|
---|
[bcdfc53] | 53 | <para>
|
---|
[7f6bf79b] | 54 | The root cause of the issue shown above was an upgrade
|
---|
[d86ea4b] | 55 | of the server software to a newer major version but leaving the
|
---|
| 56 | data files untouched. The administrator was able to recover without any
|
---|
[7f6bf79b] | 57 | loss of data.
|
---|
| 58 | </para>
|
---|
[d86ea4b] | 59 |
|
---|
[7f6bf79b] | 60 | <para>
|
---|
[d86ea4b] | 61 | Even if you are doing an install DBMS install, read through this
|
---|
| 62 | section. Tt will provide you information about how to set up backup
|
---|
[bcdfc53] | 63 | and restore procedures (at least the strategy for building
|
---|
| 64 | them) which are sufficient for your needs and for the safety
|
---|
| 65 | of your data.
|
---|
| 66 | </para>
|
---|
[8986fa9] | 67 |
|
---|
[bcdfc53] | 68 | <sect2>
|
---|
| 69 | <title>Upgrade database server packages</title>
|
---|
| 70 |
|
---|
| 71 | <para>
|
---|
[d86ea4b] | 72 | Database systems work on files which hold the database metadata and the
|
---|
| 73 | data itself. Those files are highly optimized in their internal
|
---|
| 74 | structures for use by the server software. When upgrading such server
|
---|
| 75 | software, newer server software may expect a different file format than
|
---|
| 76 | was created by previous versions. In the best case, the new software can
|
---|
| 77 | act on the old format as well—but not benefitting from newer
|
---|
| 78 | formats which might result in better performance or of other
|
---|
| 79 | improvements. It can also happen that the new server software will
|
---|
| 80 | reformat the data files automatically when starting.
|
---|
[bcdfc53] | 81 | </para>
|
---|
| 82 |
|
---|
| 83 | <para>
|
---|
[d86ea4b] | 84 | Unfortunatly, the most likly case is that the new server software
|
---|
| 85 | complains about out of date file formats and exits. When this happens
|
---|
| 86 | and you have overwritten the installed server software, you may not be
|
---|
| 87 | able to read the data files and the new software is unwilling to do so.
|
---|
[bcdfc53] | 88 | </para>
|
---|
| 89 |
|
---|
| 90 | <para>
|
---|
[d86ea4b] | 91 | Changes in data file formats usually happen at major version changes but
|
---|
| 92 | potentially can come at other times. Before upgrading the server
|
---|
| 93 | software, check the documentation if there are changes which will
|
---|
| 94 | require reformatting the database.
|
---|
[bcdfc53] | 95 | </para>
|
---|
| 96 |
|
---|
| 97 | <para>
|
---|
[d86ea4b] | 98 | Of course, if you have databases with content which is not easy to
|
---|
| 99 | rebuild, it is always a good idea to create backups of the database from
|
---|
| 100 | time to time. When upgrading the server software, it is time to run
|
---|
| 101 | another backup.
|
---|
[bcdfc53] | 102 | </para>
|
---|
| 103 |
|
---|
| 104 | <sect3>
|
---|
| 105 | <title>Upgrade by backup and restore</title>
|
---|
| 106 |
|
---|
| 107 | <note>
|
---|
| 108 | <para>
|
---|
| 109 | A backup is meaningless if there is no verified process
|
---|
| 110 | to restore the data from this backup. When running a
|
---|
| 111 | database server, you should not only create backups but you
|
---|
| 112 | should also verify that the process you designed to fullfill
|
---|
| 113 | the restore task is working properly. When you encounter a
|
---|
[d86ea4b] | 114 | problem with the restore when you urgently have to
|
---|
| 115 | rely on the backup data, it is too late—your database is
|
---|
| 116 | in danger.
|
---|
[bcdfc53] | 117 | </para>
|
---|
| 118 | </note>
|
---|
| 119 |
|
---|
| 120 | <para>
|
---|
[d86ea4b] | 121 | In general, most (all?) database server software provides some basic
|
---|
| 122 | tools to create backups of your data. Usually the backups created with
|
---|
| 123 | those tools can be read by newer versions of the software (via a
|
---|
| 124 | restore tool). Using older restore tools with newer backup data is not
|
---|
| 125 | defined and you should <emphasis>never</emphasis> blindly assume that
|
---|
| 126 | it will work. It might, but usually it doesn't.
|
---|
[bcdfc53] | 127 | </para>
|
---|
| 128 |
|
---|
| 129 | <para>
|
---|
[d86ea4b] | 130 | The easiest way to upgrade your database files is to
|
---|
[bcdfc53] | 131 | </para>
|
---|
| 132 |
|
---|
| 133 | <itemizedlist>
|
---|
| 134 | <listitem>
|
---|
| 135 | <para>Create a full database backup using the old tools.</para>
|
---|
[d86ea4b] | 136 |
|
---|
| 137 | <para>This step creates an offline copy of the database files ready
|
---|
| 138 | to be used for long term archiving, for disaster recovery, or just
|
---|
| 139 | preparation for upgrade. This offline backup consists of the full
|
---|
| 140 | one-to-one copy of the current database files or a backup of the
|
---|
| 141 | files from a certain time in history plus all journal data (that is
|
---|
| 142 | Oracle® terminology, it is called "Continuous Archiving" or
|
---|
| 143 | "write ahead log (WAL)" in Postgresql) containing information about
|
---|
| 144 | changes made to the data content. The later take less time to create
|
---|
| 145 | if the DB software provides this type of journaling as you only have
|
---|
| 146 | to save the changes after creating the last backup. The amount of
|
---|
| 147 | data to backup is much less than doing a full backup every
|
---|
| 148 | time.</para>
|
---|
| 149 |
|
---|
| 150 | <para>In terms of upgrading database server software, a full backup
|
---|
| 151 | (which can be used for subsequent incremental backups) should be
|
---|
| 152 | made, but if the amount of data is too big, an incremental backup will
|
---|
| 153 | be sufficient. Which strategy is appropriate for you depends on
|
---|
| 154 | the amount of data stored in your database (is it a few hundred table
|
---|
| 155 | rows or is it hundreds of terabytes?). A full backup of the later one
|
---|
| 156 | isn't done quickly (and we assume that the underlying system of such
|
---|
| 157 | a database is probably not on an LFS system). To close the last gap
|
---|
| 158 | to fully protect your data, create a backup of the corresponding old
|
---|
| 159 | binaries (and/or their sources) and store it along with the data
|
---|
| 160 | files to make sure that there is a fallback solution if
|
---|
| 161 | the newer software is not able to read the older data.</para>
|
---|
| 162 | </listitem>
|
---|
| 163 |
|
---|
| 164 | <listitem>
|
---|
| 165 | <para>Upgrade the server software</para>
|
---|
| 166 |
|
---|
| 167 | <para>In this step, instructions to build the database server
|
---|
| 168 | software are executed just as they are shown in subsequent sections
|
---|
| 169 | talking about the DBMs like MariaDB or Postgresql. That is, build
|
---|
| 170 | the software as usual using BLFS instructions.</para>
|
---|
| 171 | </listitem>
|
---|
| 172 |
|
---|
| 173 | <listitem>
|
---|
| 174 | <para>Restore the database by using the new tools.</para>
|
---|
| 175 |
|
---|
| 176 | <para>To restore the data, the tools of the newly installed server
|
---|
| 177 | software should be used. During the restoration process, the new
|
---|
| 178 | tools will create and/or upgrade the data files in the format the
|
---|
| 179 | software requires. It is assumed that newer software is capable of
|
---|
| 180 | reading old data.</para>
|
---|
| 181 | </listitem>
|
---|
| 182 | </itemizedlist>
|
---|
[bcdfc53] | 183 |
|
---|
| 184 | <para>
|
---|
[d86ea4b] | 185 | Since you have already have a backup procedure in place (and you
|
---|
| 186 | have tested your restore procedure, right?), this might
|
---|
| 187 | be the easiest way to upgrade as you are going to use your well known
|
---|
| 188 | processes to upgrade just as you allways do—at least in terms
|
---|
[bcdfc53] | 189 | of the backup and restore.
|
---|
| 190 | </para>
|
---|
| 191 |
|
---|
| 192 | </sect3>
|
---|
| 193 |
|
---|
| 194 | <sect3>
|
---|
[d86ea4b] | 195 | <title>Upgrade the database files by using system tools</title>
|
---|
[bcdfc53] | 196 |
|
---|
| 197 | <para>
|
---|
[d86ea4b] | 198 | Some database systems (for instance Postgresql) provide
|
---|
| 199 | a tool which can reformat (upgrade) the existing database
|
---|
[bcdfc53] | 200 | files to the new format. Since the upgrading tool has to
|
---|
| 201 | be used from the new server software (the old one cannot
|
---|
| 202 | know anything about a new file format), the old software
|
---|
| 203 | might be overwritten due to installation of the new software.
|
---|
| 204 | </para>
|
---|
| 205 |
|
---|
| 206 | <para>
|
---|
[d86ea4b] | 207 | In case you have to restore a backup (for example, running
|
---|
[bcdfc53] | 208 | the upgrade tool failed) you have to reinstall the old version
|
---|
| 209 | to get back the access to your data.
|
---|
| 210 | </para>
|
---|
| 211 |
|
---|
| 212 | <para>
|
---|
[d86ea4b] | 213 | Even though those tools might work with one of the actual database
|
---|
| 214 | files, you should create a full backup before running them. A failure
|
---|
| 215 | might result in a serious damage of the database files.
|
---|
[bcdfc53] | 216 | </para>
|
---|
| 217 |
|
---|
| 218 | </sect3>
|
---|
| 219 |
|
---|
| 220 | </sect2>
|
---|
| 221 |
|
---|
| 222 | <sect2>
|
---|
| 223 | <title>Notes for specific DBMS</title>
|
---|
[8986fa9] | 224 |
|
---|
[bcdfc53] | 225 | <sect3>
|
---|
| 226 | <title>PostgreSQL</title>
|
---|
[8986fa9] | 227 |
|
---|
[bcdfc53] | 228 | <!-- might add some advanced notes for the DBMS here -->
|
---|
| 229 |
|
---|
[d86ea4b] | 230 | <para>Upstream documentation for Backup/Restore:
|
---|
[bcdfc53] | 231 | <ulink url="https://www.postgresql.org/docs/current/backup.html"/>
|
---|
| 232 | </para>
|
---|
[8986fa9] | 233 |
|
---|
[bcdfc53] | 234 | </sect3>
|
---|
| 235 |
|
---|
| 236 | <sect3>
|
---|
| 237 | <title>MariaDB</title>
|
---|
[8986fa9] | 238 |
|
---|
[bcdfc53] | 239 | <!-- might add some advanced notes for the DBMS here -->
|
---|
| 240 |
|
---|
[d86ea4b] | 241 | <para>Upstream documentation for Backup/Restore:
|
---|
[bcdfc53] | 242 | <ulink url="https://mariadb.com/kb/en/backup-and-restore-overview/"/>
|
---|
| 243 | </para>
|
---|
[8986fa9] | 244 |
|
---|
| 245 | </sect3>
|
---|
| 246 |
|
---|
| 247 | <sect3>
|
---|
| 248 | <title>Sqlite</title>
|
---|
| 249 |
|
---|
| 250 | <!-- might add some advanced notes for the DBMS here -->
|
---|
| 251 |
|
---|
[d86ea4b] | 252 | <para>Do not underestimate <application>Sqlite</application>. It is a
|
---|
| 253 | feature rich DBMS. The main difference from the two big players above is
|
---|
| 254 | that Sqlite does not provide access via a network API. Sqlite databases
|
---|
| 255 | are files always stored on the same machine as the running program which
|
---|
| 256 | uses the database. The manipulation of data content is done via API calls
|
---|
[8986fa9] | 257 | to library functions directly within the program.</para>
|
---|
| 258 |
|
---|
[d86ea4b] | 259 | <para>In the upstream documentation you may find the following
|
---|
| 260 | useful:</para>
|
---|
[8986fa9] | 261 |
|
---|
| 262 | <para>Documentation of the sqlite3 command line tool:
|
---|
| 263 | <ulink url="https://www.sqlite.org/cli.html"/>
|
---|
| 264 | </para>
|
---|
[d86ea4b] | 265 |
|
---|
[8986fa9] | 266 | <para>Documentation of backup API calls:
|
---|
| 267 | <ulink url="https://www.sqlite.org/backup.html"/>
|
---|
| 268 | </para>
|
---|
| 269 |
|
---|
| 270 | <para>Unfortunatly, there is no dedicated chapter in the
|
---|
[d86ea4b] | 271 | upstream documentation talking about backup/restore but
|
---|
| 272 | there are several articles about it on the
|
---|
[8986fa9] | 273 | Internet. One example is shown below.</para>
|
---|
| 274 |
|
---|
[d86ea4b] | 275 | <para>Documentation for Backup/Restore:
|
---|
[8986fa9] | 276 | <ulink url="https://database.guide/backup-sqlite-database/"/>
|
---|
| 277 | </para>
|
---|
| 278 |
|
---|
[bcdfc53] | 279 | </sect3>
|
---|
| 280 |
|
---|
[8986fa9] | 281 | <sect3>
|
---|
| 282 | <title>Berkeley DB</title>
|
---|
| 283 |
|
---|
| 284 | <!-- might add some advanced notes for the DBMS here -->
|
---|
| 285 |
|
---|
| 286 | <para>Just like <application>Sqlite</application> this
|
---|
| 287 | software acts on local database files meaning there is no
|
---|
[d86ea4b] | 288 | network interface.</para>
|
---|
[8986fa9] | 289 |
|
---|
[d86ea4b] | 290 | <para>The relevant resources for Backup/Restore a Berkeley database
|
---|
[8986fa9] | 291 | are the man pages for <filename>db_dump</filename> and its
|
---|
| 292 | counterpart <filename>db_load</filename>.</para>
|
---|
| 293 |
|
---|
| 294 | </sect3>
|
---|
[bcdfc53] | 295 | </sect2>
|
---|
[8986fa9] | 296 |
|
---|
[bcdfc53] | 297 | </sect1>
|
---|