Uploaded image for project: 'ManifoldCF'
  1. ManifoldCF
  2. CONNECTORS-502

Crawling with MySQL as a backend shows signs of transactional integrity problems

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • ManifoldCF 0.6
    • ManifoldCF 1.0
    • None
    • MySQL 5.1

    Description

      When running the webcrawler-loadtests-mysql target, I got the following error:

      ERROR 2012-08-12 20:59:35,664 (Worker thread '10') - Exception tossed: Unexpected jobqueue status - record id 1344819115577, expecting active status, saw 0
      org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected jobqueue status - record id 1344819115577, expecting active status, saw 0
      	at org.apache.manifoldcf.crawler.jobs.JobQueue.updateOrHopcountRemoveRecord(JobQueue.java:800)
      	at org.apache.manifoldcf.crawler.jobs.JobManager.markDocumentHopcountRemovalMultiple(JobManager.java:2583)
      	at org.apache.manifoldcf.crawler.system.WorkerThread.processJobQueueHopcountRemovals(WorkerThread.java:1100)
      	at org.apache.manifoldcf.crawler.system.WorkerThread.processHopcountRemovalLists(WorkerThread.java:952)
      	at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:790)
      

      This indicates that although the stuffer thread thought it had put the document into the ACTIVE state, it had either never made it to the ACTIVE state or had been overridden by some other code. Either way, MySQL transactions do not seem to be working properly.

      Attachments

        Activity

          People

            kwright@metacarta.com Karl Wright
            kwright@metacarta.com Karl Wright
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: