tracker-miners3 3.2.2

[Douglas R. Reno]

New point version

[Xi Ruoyao]

Get a test failure:

ERROR: s/functional-tests/test-extraction-data/video/mp4-basic.expe (__main__.GenericExtractionTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/xry111/sources/tracker-miners-3.2.2/tests/functional-tests/extractor-generic.py", line 104, in generic_test_extraction
    sparql = fixtures.get_tracker_extract_output(extra_env,
  File "/home/xry111/sources/tracker-miners-3.2.2/tests/functional-tests/fixtures.py", line 301, in get_tracker_extract_output
    raise RuntimeError(
RuntimeError: tracker-extract returned non-zero exit code: -31
Error output:
Disallowed syscall "socket" caught in sandbox

I guess it's because something has changed in gstreamer-1.20 (or maybe I'm under some cyber attack :). If you can reproduce it, let's report to the upstream (there is no existing report for this at now).

[Xi Ruoyao]

-Dman=true still does not work. IIRC the fix is only in the "master" branch (will be released as 3.3.x).

[pierre]

Wow, got 9 failures in tests. Most of them are timeout waiting for some file to be created in ~/tracker-tests, which is not the place where I run tests! One of the errors:

---

ERROR: test_001_jpeg_title (main.WritebackImagesTest)

Traceback (most recent call last):

File "/sources/tracker-miners/tracker-miners-3.2.2/tests/functional-tests/writeback-images.py", line 96, in test_001_jpeg_title

self.writeback_test("writeback-test-1.jpeg", "image/jpeg", "nie:title")

File "/sources/tracker-miners/tracker-miners-3.2.2/tests/functional-tests/writeback-images.py", line 52, in writeback_test

path = self.prepare_test_image(self.datadir_path(filename))

File "/sources/tracker-miners/tracker-miners-3.2.2/tests/functional-tests/fixtures.py", line 469, in prepare_test_image

with self.tracker.await_insert(FILESYSTEM_GRAPH, expected, timeout=cfg.AWAIT_TIMEOUT):

File "/usr/lib/tracker-3.0/trackertestutils/storehelper.py", line 152, in exit

self.loop.run_checked()

File "/usr/lib/tracker-3.0/trackertestutils/mainloop.py", line 61, in run_checked

raise self._exception

File "/usr/lib/tracker-3.0/trackertestutils/storehelper.py", line 139, in timeout_cb

raise AwaitTimeoutException(

trackertestutils.storehelper.AwaitTimeoutException: Timeout (10s) awaiting insert of resource matching: nie:url <file:///home/pierre/tracker-tests/tmp0k_jpup3/test-monitored/writeback-test-1.jpeg>

---

Two of them are different, and occur at the same place as yours, except not with the same file:

---

ERROR: s/functional-tests/test-extraction-data/video/mov-basic.expe (main.GenericExtractionTestCase)

Traceback (most recent call last):

File "/sources/tracker-miners/tracker-miners-3.2.2/tests/functional-tests/extractor-generic.py", line 99, in generic_test_extraction

jsonld = fixtures.get_tracker_extract_output(extra_env,

File "/sources/tracker-miners/tracker-miners-3.2.2/tests/functional-tests/fixtures.py", line 312, in get_tracker_extract_output

raise RuntimeError("tracker-extract didn't return any data.\n"

RuntimeError: tracker-extract didn't return any data. Error output was: libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/dri/iHD_drv_video.so
libva info: va_openDriver() returns -1
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function vaDriverInit_1_14
[... skipping a lot of libva info similar lines]
file:///sources/tracker-miners/tracker-miners-3.2.2/tests/functional-tests/test-extraction-data/video/mov-basic.mov: No metadata or extractor modules found to handle this file

---

If I run gst-discoverer-1.0 ../tests/functional-tests/test-extraction-data/video/mov-basic.mov, I get

---

Analyzing file:///sources/tracker-miners/tracker-miners-u.2.2/build/../tests/functional-tests/test-extraction-data/video/mov-basic.mov Done discovering file:///sources/tracker-miners/tracker-miners-3.2.2/build/../tests/functional-tests/test-extraction-data/video/mov-basic.mov Missing plugins

(gstreamer|1.0|gst-discoverer-1.0|Sorensen Video 1 decoder|decoder-video/x-svq, svqversion=(int)1)

---

I get the same error when downgrading to gstreamer-1.18.5

[pierre]

Hmmm, why is it that some python files in the traceback are located in /usr/lib? Maybe this is the reason why created files are located in home...

[pierre]

Ok, I did two things to get no failures (with gstreamer-1.18.5):

• installed gstreamer-libav (was not installed on my system)
• run:

  TRACKER_TESTS_AWAIT_TIMEOUT=20 ninja test
  

(the default for TRACKER_TESTS_AWAIT_TIMEOUT is 10, and this is too short for my machine and its old hdd)

But now I get a global timeout (more than 120 s for the whole test)... let's see what is needed:

sed -i s/120/180/ tests/functional-tests/meson.build

allows the tests to just pass: total time is 172s. For security, maybe s/120/200/ or more.

[pierre]

Now all tests pass for me even with gstreamer 1.20

[Xi Ruoyao]

Replying to pierre:

Now all tests pass for me even with gstreamer 1.20

Then I guess the failure for me is because of ffmpeg-5.0...

[pierre]

Maybe or since I see:

NEW in 3.2.2 - 2022-03-06
=========================
  * Backport seccomp rules for rseq and mbind syscalls

Translations: ca, it

and the error you see is in a syscall, maybe there is a problem with libseccomp or whatever. BTW, I have kernel 5.16.14.

[pierre]

Another thing: I do not seem to need to be root to run the tests.

[pierre]

I do have libseccomp installed. Asking myself whether it was needed for tests, I actually found it is needed for building, unless -Dseccomp=false is passed: without libseccomp, and with the instructions as in the book, I get:

../meson.build:304:2: ERROR: Problem encountered: Libseccomp is necessary for sandboxed metadata extraction

I think therefore libseccomp should be moved to recommended, since I do not think it is a good idea to give up on sandboxed metadata extraction.

[pierre]

Fised at 674cb17c6 and aa36625e6.

[Douglas R. Reno]

If I remember correctly, we disabled seccomp (or suggested disabling it) on i686 machines due to issues with the rseq syscall (or was it clone3...) - this should probably be evaluated again:

" -Dseccomp=false: This option disables the seccomp system call filter. On some architectures, such as i686 and ARM, the functions that tracker-miners uses are not guarded properly, and tracker-miners will get killed with a SIGSYS as a result. "