#13167 closed enhancement (fixed)

tracker and tracker-miners 2.3.2

Reported by: Bruce Dubbs Owned by: Douglas R. Reno
Priority: normal Milestone: 9.1
Component: BOOK Version: SVN
Severity: normal Keywords:
Cc:

Description

New point version.

Change History (11)

comment:1 by Douglas R. Reno, 17 months ago

Add this into my stack before I take off for a few hours

comment:2 by Bruce Dubbs, 17 months ago

At -j4 with tests

91.0 Elapsed Time -  tracker-2.3.2
SBU=.875
1572 /usr/src/tracker/tracker-2.3.2.tar.xz SIZE (1.535 MB)
41384 kilobytes BUILD SIZE (40.414 MB)
md5sum : cf33599c3f363cfdc07fd49888051378  /usr/src/tracker/tracker-2.3.2.tar.xz

I ran the tests in xfce, but I'm not sure what is going on:

Ok:                   20
Expected Fail:         0
Fail:                 23
Unexpected Pass:       0
Skipped:               0
Timeout:               0

Example of failure output:

(tracker-store:32212): Tracker-CRITICAL *: 12:34:50.008: Cannot initialize database: no such column: Resource

gi.repository.GLib.GError: g-io-error-quark: Cannot invoke method; proxy is for the well-known name org.freedesktop.Tracker1 without an owner, and proxy was constructed with the G_DBUS_PROXY_FLAGS_DO_NOT_AUTO_START flag (0)

comment:3 by Bruce Dubbs, 17 months ago

tracker-miners: without patch -- at -j4

8.3 Elapsed Time -  tracker-miners-2.3.2
SBU=.079
3076 /usr/src/tracker-miners/tracker-miners-2.3.2.tar.xz SIZE (3.003 MB)
22552 kilobytes BUILD SIZE (22.023 MB)
md5sum : f312f2f6f9098bf6cdd0f02210e9b81d  /usr/src/tracker-miners/tracker-miners-2.3.2.tar.xz

Again tests were poor:

Ok:                   38
Expected Fail:         0
Fail:                 11
Unexpected Pass:       0
Skipped:               1
Timeout:               0

gi.repository.GLib.GError: g-io-error-quark: Cannot invoke method; proxy is for the well-known name org.freedesktop.Tracker1 without an owner, and proxy was constructed with the G_DBUS_PROXY_FLAGS_DO_NOT_AUTO_START flag (0)

comment:4 by Douglas R. Reno, 17 months ago

Owner: changed from blfs-book to Douglas R. Reno
Status: newassigned

comment:5 by Douglas R. Reno, 17 months ago

Tracker Release Notes

NEW in 2.3.2 - 2020-02-18
=========================

  * Location info for photos is now inserted into the DB. It didn't
    work before as we failed to process SPARQL "blank nodes" correctly.
  * Fix for oversensitive FTS5 index corruption detection

Translations: ms

comment:6 by Douglas R. Reno, 17 months ago

I didn't have any problems with the tests...

[0/1] Running all tests.
 1/43 functional-01-insertion                 OK       0.78 s 
 2/43 functional-02-sparql-bugs               OK       0.53 s 
 3/43 functional-03-fts-functions             OK       0.47 s 
 4/43 functional-04-group-concat              OK       0.47 s 
 5/43 functional-05-coalesce                  OK       0.42 s 
 6/43 functional-06-distance                  OK       0.52 s 
 7/43 functional-07-graph                     OK       0.47 s 
 8/43 functional-08-unique-insertions         OK       0.47 s 
 9/43 functional-09-concurrent-query          OK       0.52 s 
10/43 functional-14-signals                   OK       7.48 s 
11/43 functional-15-statistics                OK       3.47 s 
12/43 functional-16-collation                 OK       4.44 s 
13/43 functional-17-ontology-changes          OK       1.97 s 
14/43 functional-ipc-insert-or-replace        OK       0.07 s 
15/43 functional-ipc-bus-query-cancellation   OK       0.12 s 
16/43 gvdb                                    OK       0.00 s 
17/43 common-date-time                        OK       0.01 s 
18/43 common-dbus                             OK       0.01 s 
19/43 common-file-utils                       OK       0.00 s 
20/43 common-sched                            OK       0.00 s 
21/43 common-type-utils                       OK       0.01 s 
22/43 common-utils                            OK       0.00 s 
23/43 data-backup                             OK       0.42 s 
24/43 data-crc32                              OK       0.01 s 
25/43 data-db-journal                         OK       0.07 s 
26/43 data-ontology-change                    OK       0.87 s 
27/43 data-sparql-blank                       OK       0.12 s 
28/43 data-ontology                           OK       9.03 s 
29/43 data-sparql                             OK       7.39 s 
30/43 fts                                     OK       0.27 s 
31/43 miner-crawler                           OK       0.07 s 
32/43 miner-file-enumerator                   OK       0.01 s 
33/43 miner-file-system                       OK       0.01 s 
34/43 miner-indexing-tree                     OK       0.02 s 
35/43 miner-priority-queue                    OK       0.01 s 
36/43 miner-task-pool                         OK       0.01 s 
37/43 miner-thumbnailer                       OK       0.01 s 
38/43 miner-file-notifier                     OK       4.08 s 
39/43 miner-miner-fs                          OK      14.68 s 
40/43 miner-monitor                           OK      80.44 s 
41/43 tracker-resource-test                   OK       0.02 s 
42/43 tracker-sparql-test                     OK       0.22 s 
43/43 steroids                                OK       0.37 s 

Ok:                   43
Expected Fail:         0
Fail:                  0
Unexpected Pass:       0
Skipped:               0
Timeout:               0

Full log written to /sources/tracker-2.3.2/tracker-2.3.2/build/meson-logs/testlog.txt

These were run from within LXDE, in an LXTerminal window...

comment:7 by Douglas R. Reno, 17 months ago

Tracker-miners Release Notes

NEW in 2.3.2 - 2020-02-18
=========================

  * Blocked parsing of image/x-dds image files, which were previously
    being passed to GStreamer and triggering system lockups for some people
  * Fixed incorrect MIME type for .raw images, which were previously
    being passed to GStreamer and triggering system lockups for some people
  * Improved how the MP3 extractor calculates durations of variable bitrate
    MP3s, which fixes incorrect durations being shown in GNOME Music.
  * Fixed a bug in the GStreamer writeback module.
  * Fix race condition that might leave tracker-miner-fs paused after a
    writeback operation.

Translations: ms

The patch should no longer be needed.

comment:8 by Douglas R. Reno, 17 months ago

This is what I got out of tracker-miners. Going to go look at this real quick.

Ok:                   41
Expected Fail:         0
Fail:                  8
Unexpected Pass:       0
Skipped:               1
Timeout:               0

comment:9 by Douglas R. Reno, 17 months ago

A majority of the tests seem to be timing out. I modified the timeout to 120 seconds and it still seems to have problems. My conclusion so far is that the writebacktests in particular are waiting for a file to be modified, and it is unable to do so (timeouts)

[0/1] Running all tests.
 1/50 miners-common-date-time                 OK       0.02 s 
 2/50 miners-common-dbus                      OK       0.02 s 
 3/50 miners-common-file-utils                OK       0.02 s 
 4/50 miners-common-sched                     OK       0.01 s 
 5/50 miners-common-type-utils                OK       0.01 s 
 6/50 miners-common-utils                     OK       0.01 s 
 7/50 extract-extract-info                    OK       0.01 s 
 8/50 extract-module-manager                  OK       0.01 s 
 9/50 extract-guarantee                       OK       0.01 s 
10/50 extract-utils                           OK       0.02 s 
11/50 extract-xmp                             OK       0.02 s 
12/50 extract-encoding                        OK       0.01 s 
13/50 extract-exif                            OK       0.01 s 
14/50 functional/extract/audio/audio-test-1   OK       0.17 s 
15/50 functional/extract/audio/empty_albumi_song3  OK       0.17 s 
16/50 functional/extract/audio/audio-test-2   OK       0.17 s 
17/50 functional/extract/audio/audio-test-vorbis-extractor  OK       0.17 s 
18/50 functional/extract/images/test-image-1  FAIL     0.12 s (exit status 1)
19/50 functional/extract/images/xmp-loaded-1  OK       0.12 s 
20/50 functional/extract/images/test-image-3  OK       0.17 s 
21/50 functional/extract/images/corrupted-image  OK       0.17 s 
22/50 functional/extract/images/test-iptcdata-records  OK       0.12 s 
23/50 functional/extract/images/roi2          OK       0.17 s 
24/50 functional/extract/images/test-image-4  OK       0.17 s 
25/50 functional/extract/images/test-image-2  OK       0.17 s 
26/50 functional/extract/images/comment-extension-block  OK       0.12 s 
27/50 functional/extract/images/roi           OK       0.17 s 
28/50 functional/extract/playlists/playlist-test-1  OK       0.17 s 
29/50 functional/extract/office/oasis-doc     OK       0.17 s 
30/50 functional/extract/office/office-doc    OK       0.17 s 
31/50 functional/extract/office/powerpoint    OK       0.12 s 
32/50 functional/extract/office/pdf-doc       OK       0.17 s 
33/50 functional/extract/office/ps-doc        OK       0.12 s 
34/50 functional/extract/office/ps-doc-atend  OK       0.17 s 
35/50 functional/extract/office/psgz-doc      OK       0.12 s 
36/50 functional/extract/video/184505         OK       0.67 s 
37/50 functional/extract/video/video-1        OK       0.57 s 
38/50 functional/extract/video/video-2        OK       0.72 s 
39/50 functional-300-miner-basic-ops          FAIL    33.61 s (exit status 1)
40/50 functional-301-miner-resource-removal   OK       0.12 s 
41/50 functional-310-fts-basic                FAIL    34.86 s (exit status 1)
42/50 functional-311-fts-file-operations      FAIL    34.41 s (exit status 1)
43/50 functional-312-fts-stopwords            OK       4.07 s 
44/50 functional-401-extractor-flac-cuesheet  FAIL     0.97 s (exit status 1)
45/50 functional-410-extractor-decorator      OK       3.27 s 
46/50 functional-500-writeback-images         FAIL     8.52 s (exit status 1)
47/50 functional-501-writeback-image-details  FAIL    11.53 s (exit status 1)
48/50 functional-502-writeback-audio          FAIL     8.58 s (exit status 1)
49/50 functional-600-applications-camera      SKIP     0.12 s 
50/50 functional-601-applications-sync        OK      10.53 s 

Ok:                   41
Expected Fail:         0
Fail:                  8
Unexpected Pass:       0
Skipped:               1
Timeout:               0

For test-image-1, it appears that 'nao:hasTag' has not been extracted. This *may* be a problem with taglib.

AssertionError: Missing: nao:hasTag : Property 'nao:hasTag' hasn't been extracted from file 
'/sources/tracker-miners-2.3.2/tracker-miners-2.3.2/tests/functional-tests/test-extraction-data/images/test-image-1.jpg'
 (requested on '/sources/tracker-miners-2.3.2/tracker-miners-2.3.2/tests/functional-tests/test-extraction-data/images/test-image-1.expected.json')

For functional-300-miner-basic-ops, it seems that I get the same error that Bruce gets:

gi.repository.GLib.GError: g-io-error-quark: Cannot invoke method; proxy is for the well-known name org.freedesktop.Tracker1 without an owner, and proxy was constructed with the 
G_DBUS_PROXY_FLAGS_DO_NOT_AUTO_START flag (0)

For functional-310-fts-basic, I get the same error as Bruce gets. Same thing for 311-fts-file-operations. functional-401-extractor-flac-cuesheet may be because we're missing libcue (it seems that tracker-miners-3.0 will have improved detection of dependencies for tests). functional-500-writeback-images, functional-501-writeback-image-details, and functional-502-writeback-audio seem to be due to timeouts:

ERROR: test_writeback_aac (__main__.WritebackAudioTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./502-writeback-audio.py", line 58, in test_writeback_aac
    self._writeback_test(self.datadir_path('writeback-test-8.mp4'))
  File "./502-writeback-audio.py", line 43, in _writeback_test
    self.wait_for_file_change(path, initial_mtime)
  File "/sources/tracker-miners-2.3.2/tracker-miners-2.3.2/tests/functional-tests/common/utils/writebacktest.py", line 117, in wait_for_file_change
    raise Exception(
Exception: Timeout waiting for /home/renodr/tracker-tests/tmppv6pl1c4/writeback-test-8.mp4 to be updated (mtime has not changed)

At this point, I think it might be best to note that 8 tests are known to fail. The patch that I had for this package previously redesigned the testsuite to prevent these kind of timeout problems - but I'm not sure that we want to backport them again to the tracker-2.3 series (when they'll be done in tracker-3.0, which also seems to have lots of API changes)

comment:10 by Douglas R. Reno, 17 months ago

Based off the test suite results, I also notice that exif, xmp, office (libgsf), audio (gst), video (gst again), extraction, pls (totem-pl-parser), and most image files are working.

comment:11 by Douglas R. Reno, 17 months ago

Resolution: fixed
Status: assignedclosed

Fixed at r22760

Note: See TracTickets for help on using tickets.