Skip to content

Releases: tomasnorre/crawler

[RELEASE] Crawler version 11.0.4

11 Feb 07:28
Compare
Choose a tag to compare

Fixed

  • Crawler could not index frontend because of trustedHostPattern mismatch
  • Fatal PHP error is thrown in the backend crawler log
  • Prevent buildQueue from creating entries for non-localized pages
  • CrawlerController checking if URL is correct ObjectType fixed

Deprecated

Functions & Properties

  • ConfigurationRepository->getCrawlerConfigurationRecords()
  • ProcessRepository->findByProcessId()
  • QueueRepository->countAllUnassignedPendingItems()
  • QueueRepository->countPendingItemsGroupedByConfigurationKey()
  • QueueRepository->getSetIdWithUnprocessedEntries()
  • QueueRepository->getTotalQueueEntriesByConfiguration()
  • QueueRepository->getLastProcessedEntriesTimestamps()
  • QueueRepository->getLastProcessedEntries()
  • QueueRepository->getPerformanceData()
  • QueueRepository->isPageInQueueTimed()
  • QueueRepository->getAvailableSets()
  • QueueRepository->findByQueueId()

New Contributors

[RELEASE] Crawler version 11.0.3

28 Jan 13:41
Compare
Choose a tag to compare

Added

  • PHP 8.1 compatibility

Fixed

  • PHP8 undefined array key when creating/updating/saving configuration record
  • RestrictedPages is getting crawled when Crawler Configuration contains the belonging UserGroup

Contributors

@brotkrueml
@tomasnorre

[RELEASE] Crawler version 11.0.2

11 Jan 19:11
Compare
Choose a tag to compare

Added

  • Docs: Troubleshooting - Site based baseVariants not used
  • Docs: Example on how to exclude pages recursively from being crawled
  • Docs: Respecting Categories in News to prevent double indexing
  • Docs: Set correct main branch name

Fixed

  • DateTime parse error in backend module when displaying log
  • PHP8 undefined array keys in CrawlerController, ConfigurationService and middlewares
  • Text not displayed in confirm dialog in backend module

New Contributors

[RELEASE] Crawler version 11.0.1

01 Nov 14:10
8ee8bc4
Compare
Choose a tag to compare

Changed

  • Use more expressive message for excluded doktypes when building queue

Fixed

  • Set result_data back to json_encode($result) to ensure correct result handling in backend
  • Removes QueueRepository from FrontendUserAuthenticator Middleware, to ensure FE plugins can be rendered
  • PHP 8.0 compatibility problems. Undefined indexes resolved
  • Change requests to write to Header to ensure StaticFileCache is working
  • Switched back to fetch() instead of fetchAssociative() to keep support for legacy TYPO3 installations

[RELEASE] Crawler version 11.0.0

22 Oct 13:28
Compare
Choose a tag to compare

Only added functionality is added here, please see the
https://github.com/tomasnorre/crawler/blob/main/CHANGELOG.md
for more changes.

Added

  • Support for TYPO3 11 LTS
  • PHP 8.0 Support
  • Display error messages in buildQueue command in CLI
  • PSR-14 ModifySkipPageEvent
  • PSR-14 AfterUrlCrawledEvent
  • PSR-14 InvokeQueueChangeEvent
  • PSR-14 AfterUrlCrawledEvent
  • PSR-14 BeforeQueueItemAddedEvent
  • PSR-14 AfterQueueItemAddedEvent

[RELEASE] Crawler version 10.0.2

16 Sep 14:11
Compare
Choose a tag to compare

Added

  • Replace in composer for aoepeople/crawler

Changed

  • Changed composer package name to tomasnorre/crawler

[RELEASE] Crawler version 10.0.1

27 Aug 13:32
Compare
Choose a tag to compare
  • Fixed
    Exclude Pages in Crawler configs accepts e.g. 6+3 again, page id 6 & depth 3

[RELEASE] Crawler version 10.0.0

03 Aug 14:59
Compare
Choose a tag to compare
  • Added
    Added information about indexed_search money pool

  • Fixed
    Add processId to finished QueueEntries to ensure the process will been seen as successful instead of cancelled.
    Fix type error when exclude subcfg is not defined
    Sites/URLs get crawled but not indexed @twestner

  • Removed Properties
    !!! CrawlerController->tableName
    !!! ConfigurationRepository->tableName
    !!! ProcessRepository->tableName
    !!! QueueRepository->tableName

[RELEASE] Crawler version 9.2.4

10 Apr 17:14
Compare
Choose a tag to compare

Fixes:

  • Date parsing for immediate queue entries

[RELEASE] Crawler version 9.2.3

10 Apr 17:14
Compare
Choose a tag to compare
[RELEASE] Crawler version 9.2.3