Add directory listing scraper for S3/GCS/local filesystems

[Copilot]

Implements scraper that creates config items from directory metadata across S3, GCS, and local filesystems. Uses flanksource/artifacts library for unified filesystem abstraction, pattern matching from canary-checker/folder.go.

Implementation

• api/v1/folder.go: Folder config type with S3/GCS/local connection support, filter options (age, size, regex, glob), recursive scanning
• scrapers/folder/folder.go: Scraper using artifacts.GetFSForConnection() for cross-platform FS access, generates File::Metadata and Folder::Listing config items with metadata (name, size, modTime, mode)
• Registry: Added to scrapers/common.go scraper list

Config items structure

{
  "name": "backup.tar.gz",
  "path": "/data/backups",
  "size": 1073741824,
  "modTime": "2024-01-15T10:30:00Z",
  "isDir": false,
  "mode": "-rw-r--r--"
}

Example usage

apiVersion: configs.flanksource.com/v1
kind: ScrapeConfig
spec:
  folder:
    - s3:
        bucket: logs-bucket
        region: us-west-2
      path: "application-logs/"
      filter:
        glob: "*.log"
        minAge: "24h"
        minSize: 1024

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Directory Listing Scraper</issue_title>
<issue_description>- i.e. create config items from S3/file/etc directory listing metadata - Use https://github.com/flanksource/canary-checker/blob/master/checks/folder.go as baseline and flanksource/artifacts library </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Directory Listing Scraper #1897

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}