xapian-core-1.2.21

[Fernando de Oliveira]

​http://oligarchy.co.uk/xapian/1.2.21/xapian-core-1.2.21.tar.xz

​http://lists.xapian.org/pipermail/xapian-discuss/2015-May/009201.html

SHA256: 63f48758fbd13fa8456dd4cf9bf3ec35a096e4290f14a51ac7df23f78c162d3f

​http://oligarchy.co.uk/xapian/1.2.21/xapian-core-1.2.21.tar.xz.asc

​http://xapian.org/docs/xapian-core-1.2.21/NEWS

Xapian-core 1.2.21 (2015-05-20):

API:

• QueryParser: Extend the set of characters allowed in the start of a
  range to be anything except for '(' and characters <= ' '.  This better
  matches what's accepted for a range end (anything except for ')' and
  characters <= ' ').  Reported by Jani Nikula.

matcher:

• Reimplement OP_PHRASE for non-exact phrases.  The previous
  implementation was buggy, giving both false positives and false
  negatives in rare cases when three or more terms were involved.  Fixes
  #653, reported by Jean-Francois Dockes.

• Reimplement OP_NEAR - the new implementation consistently requires the
  terms to occur at different positions, and fixes some previously missed
  matches.

• Fix a reversed check for picking the shorter position list for an exact
  phrase of two terms.  The difference this makes isn't dramatic, but can
  be measured (at least with cachegrind).  Thanks to kbwt for spotting
  this.

• When matching an exact phrase, if a term doesn't occur where we want,
  use its actual position to advance the anchor term, rather than just
  checking the next position of the anchor term.

brass backend:

• Fix cursor versioning to consider cancel() and reopen() as events where
  the cursor version may need incrementing, and flag the current cursor
  version as used when a cursor is rebuilt.  Fixes #675, reported by
  Germán M. Bravo.

• Avoid using file descriptions < 3 for writable database tables, as it
  risks corruption if some code in the same process tries to write to
  stdout or stderr without realising it is closed.  (Partly addresses
  #651)

chert backend:

• Fix cursor versioning to consider cancel() and reopen() as events where
  the cursor version may need incrementing, and flag the current cursor
  version as used when a cursor is rebuilt.  Fixes #675, reported by
  Germán M. Bravo.

• Avoid using file descriptions < 3 for writable database tables, as it
  risks corruption if some code in the same process tries to write to
  stdout or stderr without realising it is closed.  (Partly addresses
  #651)

flint backend:

• Fix cursor versioning to consider cancel() and reopen() as events where
  the cursor version may need incrementing, and flag the current cursor
  version as used when a cursor is rebuilt.  Fixes #675, reported by
  Germán M. Bravo.

remote backend:

• Fix sort by value when multiple databases are in use and one or more
  are remote.  This change necessitated a minor version bump in the
  remote protocol.  Fixes #674, reported by Dylan Griffith.  If you are
  upgrading a live system which uses the remote backend, upgrade the
  servers before the clients.

build system:

• The compiler ABI check in the public API headers now issues a warning
  (instead of an error) for an ABI mismatch for ABI versions 2 and later
  (which means GCC >= 3.4).  The changes in these ABI versions are bug
  fixes for corner cases, so there's a good chance of things working -
  e.g. building xapian-bindings with GCC 5.1 (which defaults to ABI
  version 8) against xapian-core built with GCC 4.9 (which defaults to
  ABI version 2) seems to work OK.  A warning is still useful as a clue
  to what is going on if linking fails due to a missing symbol.

• xapian-config,xapian-core.pc: When compiling with xlC on AIX, the
  reported --cxxflags/--cflags now include -D_LARGE_FILES=1 as this is
  defined for the library, and defining it changes the ABI of std::string
  with this compiler, so it must also be defined when building code using
  the Xapian API.

• xapian-core.pc: Include --enable-runtime-pseudo-reloc in --libs output
  for mingw and cygwin, like xapian-config does.

• xapian-core.pc: Fix include directory reported by `pkg-config
  --cflags`.  This bug was harmless if xapian-core was installed to a
  directory which was on the default header search path (such as
  /usr/include).

• xapian-config: Fix typo so cached result of test in is_uninstalled() is
  actually used on subsequent calls.  Fixes #676, reported (with patch)
  by Ryan Schmidt.

• configure: Changes in 1.2.19 broke the custom macro we use to probe for
  supported compiler flags such that the flags never got used.  This
  release fixes this problem.

• configure: Set default value for AUTOM4TE before AC_OUTPUT so the
  default will actually get used.  Only relevant when building in
  maintainer mode (e.g. from git).

• soaktest: Link with libtool's '-no-install' or '-no-fast-install', like
  we already do for other test programs, which means that libtool doesn't
  need to generate shell script wrappers for them on most platforms.

documentation:

• API documentation: Minor wording tweaks and formatting improvements.

• docs/deprecation.rst: Add deprecation of omindex
  --preserve-nonduplicates which happened in 1.2.4.

• HACKING: Update URL.

• HACKING: libtool 2.4.6 is now used for bootstrapping snapshots and
  releases.

tools:

• xapian-compact: Make sure we open all the tables of input databases at
  the same revision.  (Fixes #649)

• xapian-metadata: Add 'list' subcommand to list all the metadata keys.

• xapian-replicate: Fix connection timeout to be 10 seconds rather than
  10000 seconds (the incorrect timeout has been the case since 1.2.3).

• xapian-replicate: Set SO_KEEPALIVE for xapian-replicate's connection to
  the master, and add command line option to allow setting socket-level
  timeouts (SO_RCVTIMEO and SO_SNDTIMEO) on platforms that support them.
  Fixes #546, reported by nkvoll.

• xapian-replicate-server: Avoid potentially reading uninitialised data
  if a changeset file is truncated.

portability:

• Add spaces between literal strings and macros which expand to literal
  strings for C++11 compatibility.

• ValueCountMatchSpy::top_values_begin(): Fix the comparison function not
  to return true for two equal elements, which manifests as incorrect
  sorting in some cases when using clang's libc++ (which recent OS X
  versions do).

• apitest: The adddoc5 testcase fails under clang due to an exception
  handling bug, so just #ifdef out the problematic part of the testcase
  when building with clang for now.

• Fix clang warnings on OS X.  Reported by Germán M. Bravo.

• Fix examples to build with IBM's xlC compiler on AIX - they were
  failing due to _LARGE_FILES being defined for the library build but not
  for the examples, and defining this changes the ABI of std::string with
  this compiler.

• configure: Improve the probe for whether the test harness can use RTTI
  to work for IBM's xlC compiler (which defaults to not generating RTTI).

• Fix to build with Sun's C++ compiler.

• Use F_DUPFD where available to dup to a file descriptor which is >= 2,
  rather than calling dup() until we get one.

• When unserialising a double, avoid reading one byte past the end of the
  serialised value.  In practice this was harmless on most platforms, as
  dbl_max_mantissa is 255 for IEEE-754 format doubles, and at least GCC's
  std::string keeps the buffer nul-terminated.  Reported by Germán M.
  Bravo in github PR#67.

• When unserialising a double, add missing cast to unsigned char when we
  check if the value will fit in the double type.  On machines with
  IEEE-754 doubles (which is most current platforms) this happened to
  work OK before.  It would also have been fine on machines where char is
  unsigned by default.

• Fix incorrect use of "delete" which should be "delete []".  This is
  undefined behaviour in C++, though the type is POD, so in practice this
  probably worked OK on many platforms.

debug code:

• Fix some overly strict assertions in flint, which caused apitest's
  cursordelbug1 to fail with assertions on.

[Fernando de Oliveira]

I'm having a small problem with the tests and will ignore it, when updating.

However, if somebody can give a clue or improve the page, perhaps it could be good.

Problem is that 1 of 6 tests fails.

I ran some greps:

FAIL: apitest
PASS: internaltest
PASS: stemtest
PASS: queryparsertest
PASS: termgentest
PASS: unittest
=======================================
1 of 6 tests failed
Please report to http://xapian.org/bugs
=======================================

When inspected, only problem I found was

...
./apitest backend inmemory: All 226 tests passed, 8 skipped.
Running tests with backend "brass"...
...
Running test: deldoc4... Mismatched free() / delete / delete []
...
./apitest backend brass: 301 tests passed, 1 failed, 2 skipped.
...
./apitest total: 2992 tests passed, 1 failed, 45 skipped.
...

Therefore, it is not really 1 in 6, but 1 in more than 2992, which I consider as negligible.

This mismatch has been related to bug in the code:

​http://cs.ecs.baylor.edu/~donahoo/tools/valgrind/messages.html

 In C++ it's important to deallocate memory in a way compatible with how it was allocated. The deal is:

    If allocated with malloc, calloc, realloc, valloc or memalign, you must deallocate with free.
    If allocated with new[], you must deallocate with delete[].
    If allocated with new, you must deallocate with delete. 

However, when run "by hand", I got:

(by memory)

xapian-core-1.2.21/tests ]$ ./apitest -b brass
Running tests with backend "brass"...
...
Running test: deldoc4... ok
...
./apitest backend brass: 302 tests passed, 0 failed, 2 skipped.

(/by memory)

and (copy/paste):

xapian-core-1.2.21/tests ]$ ./apitest -v -b brass deldoc4
Running tests with backend "brass"...
Running test: deldoc4... ok
./apitest backend brass: All 1 tests passed.

[Fernando de Oliveira]

Fixed at r16025.