source: server/databases/upgradedb.xml@ d86ea4b

11.1 lazarus trunk upgradedb xry111/intltool xry111/test-20220226
Last change on this file since d86ea4b was d86ea4b, checked in by Bruce Dubbs <bdubbs@…>, 5 months ago

Editorial changes to section about ubrading DBMS

  • Property mode set to 100644
File size: 12.1 KB
1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "" [
4 <!ENTITY % general-entities SYSTEM "../../general.ent">
5 %general-entities;
8<sect1 id="upgradedb" xreflabel="Upgrade Database">
9 <?dbhtml filename="upgradedb.html"?>
11 <sect1info>
12 <date>$Date$</date>
13 </sect1info>
15 <title>Important notes about upgrading Database Server Software</title>
17 <note><para>This section is about reinstalling database software
18 when an existing database is in use. It is not applicable for
19 initial installations or if there is no existing database for
20 the package being updated.</para></note>
22 <para>Lets start this chapter with a dramatic screenshot of an issue that
23 really happened. This issue will not occur if you are going to install the
24 software the first time:</para>
26<screen>$ sudo systemctl status postgresql
27-- postgresql.service - PostgreSQL database server
28 Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
29 Active: failed (Result: exit-code) since Tue 2021-10-26 17:11:53 CDT; 2min 49s ago
30 Process: 17336 ExecStart=/usr/bin/pg_ctl -s -D ${PGROOT}/data start -w -t 120 (code=exited, status=1/FAILURE)
31 CPU: 7ms
33Oct 26 17:11:53 SVRNAME systemd[1]: Starting PostgreSQL database server...
34Oct 26 17:11:53 SRVNAME postgres[17338]: 2021-10-26 17:11:53.420 CDT [17338] FATAL:
35 database files are incompatible with server
36Oct 26 17:11:53 SRVNAME postgres[17338]: 2021-10-26 17:11:53.420 CDT [17338] DETAIL:
37 The data directory was initialized by PostgreSQL version 13,
38 which is not compatible with this version 14.0.
39Oct 26 17:11:53 SRVNAME postgres[17336]: pg_ctl: could not start server
40Oct 26 17:11:53 SRVNAME postgres[17336]: Examine the log output.
41Oct 26 17:11:53 SRVNAME systemd[1]: postgresql.service: Control process exited, code=exited, status=1/FAILURE
42Oct 26 17:11:53 SRVNAME systemd[1]: postgresql.service: Failed with result 'exit-code'.
43Oct 26 17:11:53 SRVNAME systemd[1]: Failed to start PostgreSQL database server.</screen>
45 <para>
46 To avoid situations like the one above finding your database server
47 software refusing to start, read the following thoughts about how to
48 upgrade a DBMS (Database Management System) prior to actually doing the
49 upgrade.
50 </para>
52 <para>
53 The root cause of the issue shown above was an upgrade
54 of the server software to a newer major version but leaving the
55 data files untouched. The administrator was able to recover without any
56 loss of data.
57 </para>
59 <para>
60 Even if you are doing an install DBMS install, read through this
61 section. Tt will provide you information about how to set up backup
62 and restore procedures (at least the strategy for building
63 them) which are sufficient for your needs and for the safety
64 of your data.
65 </para>
67 <sect2>
68 <title>Upgrade database server packages</title>
70 <para>
71 Database systems work on files which hold the database metadata and the
72 data itself. Those files are highly optimized in their internal
73 structures for use by the server software. When upgrading such server
74 software, newer server software may expect a different file format than
75 was created by previous versions. In the best case, the new software can
76 act on the old format as well&mdash;but not benefitting from newer
77 formats which might result in better performance or of other
78 improvements. It can also happen that the new server software will
79 reformat the data files automatically when starting.
80 </para>
82 <para>
83 Unfortunatly, the most likly case is that the new server software
84 complains about out of date file formats and exits. When this happens
85 and you have overwritten the installed server software, you may not be
86 able to read the data files and the new software is unwilling to do so.
87 </para>
89 <para>
90 Changes in data file formats usually happen at major version changes but
91 potentially can come at other times. Before upgrading the server
92 software, check the documentation if there are changes which will
93 require reformatting the database.
94 </para>
96 <para>
97 Of course, if you have databases with content which is not easy to
98 rebuild, it is always a good idea to create backups of the database from
99 time to time. When upgrading the server software, it is time to run
100 another backup.
101 </para>
103 <sect3>
104 <title>Upgrade by backup and restore</title>
106 <note>
107 <para>
108 A backup is meaningless if there is no verified process
109 to restore the data from this backup. When running a
110 database server, you should not only create backups but you
111 should also verify that the process you designed to fullfill
112 the restore task is working properly. When you encounter a
113 problem with the restore when you urgently have to
114 rely on the backup data, it is too late&mdash;your database is
115 in danger.
116 </para>
117 </note>
119 <para>
120 In general, most (all?) database server software provides some basic
121 tools to create backups of your data. Usually the backups created with
122 those tools can be read by newer versions of the software (via a
123 restore tool). Using older restore tools with newer backup data is not
124 defined and you should <emphasis>never</emphasis> blindly assume that
125 it will work. It might, but usually it doesn't.
126 </para>
128 <para>
129 The easiest way to upgrade your database files is to
130 </para>
132 <itemizedlist>
133 <listitem>
134 <para>Create a full database backup using the old tools.</para>
136 <para>This step creates an offline copy of the database files ready
137 to be used for long term archiving, for disaster recovery, or just
138 preparation for upgrade. This offline backup consists of the full
139 one-to-one copy of the current database files or a backup of the
140 files from a certain time in history plus all journal data (that is
141 Oracle&reg; terminology, it is called "Continuous Archiving" or
142 "write ahead log (WAL)" in Postgresql) containing information about
143 changes made to the data content. The later take less time to create
144 if the DB software provides this type of journaling as you only have
145 to save the changes after creating the last backup. The amount of
146 data to backup is much less than doing a full backup every
147 time.</para>
149 <para>In terms of upgrading database server software, a full backup
150 (which can be used for subsequent incremental backups) should be
151 made, but if the amount of data is too big, an incremental backup will
152 be sufficient. Which strategy is appropriate for you depends on
153 the amount of data stored in your database (is it a few hundred table
154 rows or is it hundreds of terabytes?). A full backup of the later one
155 isn't done quickly (and we assume that the underlying system of such
156 a database is probably not on an LFS system). To close the last gap
157 to fully protect your data, create a backup of the corresponding old
158 binaries (and/or their sources) and store it along with the data
159 files to make sure that there is a fallback solution if
160 the newer software is not able to read the older data.</para>
161 </listitem>
163 <listitem>
164 <para>Upgrade the server software</para>
166 <para>In this step, instructions to build the database server
167 software are executed just as they are shown in subsequent sections
168 talking about the DBMs like MariaDB or Postgresql. That is, build
169 the software as usual using BLFS instructions.</para>
170 </listitem>
172 <listitem>
173 <para>Restore the database by using the new tools.</para>
175 <para>To restore the data, the tools of the newly installed server
176 software should be used. During the restoration process, the new
177 tools will create and/or upgrade the data files in the format the
178 software requires. It is assumed that newer software is capable of
179 reading old data.</para>
180 </listitem>
181 </itemizedlist>
183 <para>
184 Since you have already have a backup procedure in place (and you
185 have tested your restore procedure, right?), this might
186 be the easiest way to upgrade as you are going to use your well known
187 processes to upgrade just as you allways do&mdash;at least in terms
188 of the backup and restore.
189 </para>
191 </sect3>
193 <sect3>
194 <title>Upgrade the database files by using system tools</title>
196 <para>
197 Some database systems (for instance Postgresql) provide
198 a tool which can reformat (upgrade) the existing database
199 files to the new format. Since the upgrading tool has to
200 be used from the new server software (the old one cannot
201 know anything about a new file format), the old software
202 might be overwritten due to installation of the new software.
203 </para>
205 <para>
206 In case you have to restore a backup (for example, running
207 the upgrade tool failed) you have to reinstall the old version
208 to get back the access to your data.
209 </para>
211 <para>
212 Even though those tools might work with one of the actual database
213 files, you should create a full backup before running them. A failure
214 might result in a serious damage of the database files.
215 </para>
217 </sect3>
219 </sect2>
221 <sect2>
222 <title>Notes for specific DBMS</title>
224 <sect3>
225 <title>PostgreSQL</title>
227 <!-- might add some advanced notes for the DBMS here -->
229 <para>Upstream documentation for Backup/Restore:
230 <ulink url=""/>
231 </para>
233 </sect3>
235 <sect3>
236 <title>MariaDB</title>
238 <!-- might add some advanced notes for the DBMS here -->
240 <para>Upstream documentation for Backup/Restore:
241 <ulink url=""/>
242 </para>
244 </sect3>
246 <sect3>
247 <title>Sqlite</title>
249 <!-- might add some advanced notes for the DBMS here -->
251 <para>Do not underestimate <application>Sqlite</application>. It is a
252 feature rich DBMS. The main difference from the two big players above is
253 that Sqlite does not provide access via a network API. Sqlite databases
254 are files always stored on the same machine as the running program which
255 uses the database. The manipulation of data content is done via API calls
256 to library functions directly within the program.</para>
258 <para>In the upstream documentation you may find the following
259 useful:</para>
261 <para>Documentation of the sqlite3 command line tool:
262 <ulink url=""/>
263 </para>
265 <para>Documentation of backup API calls:
266 <ulink url=""/>
267 </para>
269 <para>Unfortunatly, there is no dedicated chapter in the
270 upstream documentation talking about backup/restore but
271 there are several articles about it on the
272 Internet. One example is shown below.</para>
274 <para>Documentation for Backup/Restore:
275 <ulink url=""/>
276 </para>
278 </sect3>
280 <sect3>
281 <title>Berkeley DB</title>
283 <!-- might add some advanced notes for the DBMS here -->
285 <para>Just like <application>Sqlite</application> this
286 software acts on local database files meaning there is no
287 network interface.</para>
289 <para>The relevant resources for Backup/Restore a Berkeley database
290 are the man pages for <filename>db_dump</filename> and its
291 counterpart <filename>db_load</filename>.</para>
293 </sect3>
294 </sect2>
Note: See TracBrowser for help on using the repository browser.