source: server/databases/upgradedb.xml@ 5f729f8

11.2 plabs/python-mods trunk xry111/soup3
Last change on this file since 5f729f8 was 5f729f8, checked in by Douglas R. Reno <renodr@…>, 4 months ago

Various typo fixes

  • Property mode set to 100644
File size: 12.1 KB
1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "" [
4 <!ENTITY % general-entities SYSTEM "../../general.ent">
5 %general-entities;
8<sect1 id="upgradedb" xreflabel="Upgrade Database">
9 <?dbhtml filename="upgradedb.html"?>
11 <sect1info>
12 <date>$Date$</date>
13 </sect1info>
15 <title>Important notes about upgrading Database Server Software</title>
17 <note><para>This section is about reinstalling database software
18 when an existing database is in use. It is not applicable for
19 initial installations or if there is no existing database for
20 the package being updated, but users should read through it
21 to become aware of issues that can arise in the future.</para></note>
23 <para>Lets start this chapter with a dramatic screenshot of an issue that
24 really happened. This issue will not occur if you are going to install the
25 software the first time:</para>
27<screen>$ sudo systemctl status postgresql
28-- postgresql.service - PostgreSQL database server
29 Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
30 Active: failed (Result: exit-code) since Tue 2021-10-26 17:11:53 CDT; 2min 49s ago
31 Process: 17336 ExecStart=/usr/bin/pg_ctl -s -D ${PGROOT}/data start -w -t 120 (code=exited, status=1/FAILURE)
32 CPU: 7ms
34Oct 26 17:11:53 SVRNAME systemd[1]: Starting PostgreSQL database server...
35Oct 26 17:11:53 SRVNAME postgres[17338]: 2021-10-26 17:11:53.420 CDT [17338] FATAL:
36 database files are incompatible with server
37Oct 26 17:11:53 SRVNAME postgres[17338]: 2021-10-26 17:11:53.420 CDT [17338] DETAIL:
38 The data directory was initialized by PostgreSQL version 13,
39 which is not compatible with this version 14.0.
40Oct 26 17:11:53 SRVNAME postgres[17336]: pg_ctl: could not start server
41Oct 26 17:11:53 SRVNAME postgres[17336]: Examine the log output.
42Oct 26 17:11:53 SRVNAME systemd[1]: postgresql.service: Control process exited, code=exited, status=1/FAILURE
43Oct 26 17:11:53 SRVNAME systemd[1]: postgresql.service: Failed with result 'exit-code'.
44Oct 26 17:11:53 SRVNAME systemd[1]: Failed to start PostgreSQL database server.</screen>
46 <para>
47 To avoid situations like the one above finding your database server
48 software refusing to start, read the following thoughts about how to
49 upgrade a DBMS (Database Management System) prior to actually doing the
50 upgrade.
51 </para>
53 <para>
54 The root cause of the issue shown above was an upgrade
55 of the server software to a newer major version but leaving the
56 data files untouched. The administrator was able to recover without any
57 loss of data.
58 </para>
60 <para>
61 Even if you are doing an install DBMS install, read through this
62 section. Tt will provide you information about how to set up backup
63 and restore procedures (at least the strategy for building
64 them) which are sufficient for your needs and for the safety
65 of your data.
66 </para>
68 <sect2>
69 <title>Upgrade database server packages</title>
71 <para>
72 Database systems work on files which hold the database metadata and the
73 data itself. Those files are highly optimized in their internal
74 structures for use by the server software. When upgrading such server
75 software, newer server software may expect a different file format than
76 was created by previous versions. In the best case, the new software can
77 act on the old format as well&mdash;but not benefitting from newer
78 formats which might result in better performance or of other
79 improvements. It can also happen that the new server software will
80 reformat the data files automatically when starting.
81 </para>
83 <para>
84 Unfortunately, the most likely case is that the new server software
85 complains about out of date file formats and exits. When this happens
86 and you have overwritten the installed server software, you may not be
87 able to read the data files and the new software is unwilling to do so.
88 </para>
90 <para>
91 Changes in data file formats usually happen at major version changes but
92 potentially can come at other times. Before upgrading the server
93 software, check the documentation if there are changes which will
94 require reformatting the database.
95 </para>
97 <para>
98 Of course, if you have databases with content which is not easy to
99 rebuild, it is always a good idea to create backups of the database from
100 time to time. When upgrading the server software, it is time to run
101 another backup.
102 </para>
104 <sect3>
105 <title>Upgrade by backup and restore</title>
107 <note>
108 <para>
109 A backup is meaningless if there is no verified process
110 to restore the data from this backup. When running a
111 database server, you should not only create backups but you
112 should also verify that the process you designed to fulfill
113 the restore task is working properly. When you encounter a
114 problem with the restore when you urgently have to
115 rely on the backup data, it is too late&mdash;your database is
116 in danger.
117 </para>
118 </note>
120 <para>
121 In general, most (all?) database server software provides some basic
122 tools to create backups of your data. Usually the backups created with
123 those tools can be read by newer versions of the software (via a
124 restore tool). Using older restore tools with newer backup data is not
125 defined and you should <emphasis>never</emphasis> blindly assume that
126 it will work. It might, but usually it doesn't.
127 </para>
129 <para>
130 The easiest way to upgrade your database files is to
131 </para>
133 <itemizedlist>
134 <listitem>
135 <para>Create a full database backup using the old tools.</para>
137 <para>This step creates an offline copy of the database files ready
138 to be used for long term archiving, for disaster recovery, or just
139 preparation for upgrade. This offline backup consists of the full
140 one-to-one copy of the current database files or a backup of the
141 files from a certain time in history plus all journal data (that is
142 Oracle&reg; terminology, it is called "Continuous Archiving" or
143 "write ahead log (WAL)" in Postgresql) containing information about
144 changes made to the data content. The later take less time to create
145 if the DB software provides this type of journaling as you only have
146 to save the changes after creating the last backup. The amount of
147 data to backup is much less than doing a full backup every
148 time.</para>
150 <para>In terms of upgrading database server software, a full backup
151 (which can be used for subsequent incremental backups) should be
152 made, but if the amount of data is too big, an incremental backup will
153 be sufficient. Which strategy is appropriate for you depends on
154 the amount of data stored in your database (is it a few hundred table
155 rows or is it hundreds of terabytes?). A full backup of the later one
156 isn't done quickly (and we assume that the underlying system of such
157 a database is probably not on an LFS system). To close the last gap
158 to fully protect your data, create a backup of the corresponding old
159 binaries (and/or their sources) and store it along with the data
160 files to make sure that there is a fallback solution if
161 the newer software is not able to read the older data.</para>
162 </listitem>
164 <listitem>
165 <para>Upgrade the server software</para>
167 <para>In this step, instructions to build the database server
168 software are executed just as they are shown in subsequent sections
169 talking about the DBMs like MariaDB or Postgresql. That is, build
170 the software as usual using BLFS instructions.</para>
171 </listitem>
173 <listitem>
174 <para>Restore the database by using the new tools.</para>
176 <para>To restore the data, the tools of the newly installed server
177 software should be used. During the restoration process, the new
178 tools will create and/or upgrade the data files in the format the
179 software requires. It is assumed that newer software is capable of
180 reading old data.</para>
181 </listitem>
182 </itemizedlist>
184 <para>
185 Since you have already have a backup procedure in place (and you
186 have tested your restore procedure, right?), this might
187 be the easiest way to upgrade as you are going to use your well known
188 processes to upgrade just as you always do&mdash;at least in terms
189 of the backup and restore.
190 </para>
192 </sect3>
194 <sect3>
195 <title>Upgrade the database files by using system tools</title>
197 <para>
198 Some database systems (for instance Postgresql) provide
199 a tool which can reformat (upgrade) the existing database
200 files to the new format. Since the upgrading tool has to
201 be used from the new server software (the old one cannot
202 know anything about a new file format), the old software
203 might be overwritten due to installation of the new software.
204 </para>
206 <para>
207 In case you have to restore a backup (for example, running
208 the upgrade tool failed) you have to reinstall the old version
209 to get back the access to your data.
210 </para>
212 <para>
213 Even though those tools might work with one of the actual database
214 files, you should create a full backup before running them. A failure
215 might result in a serious damage of the database files.
216 </para>
218 </sect3>
220 </sect2>
222 <sect2>
223 <title>Notes for specific DBMS</title>
225 <sect3>
226 <title>PostgreSQL</title>
228 <!-- might add some advanced notes for the DBMS here -->
230 <para>Upstream documentation for Backup/Restore:
231 <ulink url=""/>
232 </para>
234 </sect3>
236 <sect3>
237 <title>MariaDB</title>
239 <!-- might add some advanced notes for the DBMS here -->
241 <para>Upstream documentation for Backup/Restore:
242 <ulink url=""/>
243 </para>
245 </sect3>
247 <sect3>
248 <title>Sqlite</title>
250 <!-- might add some advanced notes for the DBMS here -->
252 <para>Do not underestimate <application>Sqlite</application>. It is a
253 feature rich DBMS. The main difference from the two big players above is
254 that Sqlite does not provide access via a network API. Sqlite databases
255 are files always stored on the same machine as the running program which
256 uses the database. The manipulation of data content is done via API calls
257 to library functions directly within the program.</para>
259 <para>In the upstream documentation you may find the following
260 useful:</para>
262 <para>Documentation of the sqlite3 command line tool:
263 <ulink url=""/>
264 </para>
266 <para>Documentation of backup API calls:
267 <ulink url=""/>
268 </para>
270 <para>Unfortunately, there is no dedicated chapter in the
271 upstream documentation talking about backup/restore but
272 there are several articles about it on the
273 Internet. One example is shown below.</para>
275 <para>Documentation for Backup/Restore:
276 <ulink url=""/>
277 </para>
279 </sect3>
281 <sect3>
282 <title>Berkeley DB</title>
284 <!-- might add some advanced notes for the DBMS here -->
286 <para>Just like <application>Sqlite</application> this
287 software acts on local database files meaning there is no
288 network interface.</para>
290 <para>The relevant resources for Backup/Restore a Berkeley database
291 are the man pages for <filename>db_dump</filename> and its
292 counterpart <filename>db_load</filename>.</para>
294 </sect3>
295 </sect2>
Note: See TracBrowser for help on using the repository browser.