v0.6.0 - MongoDB Full-Text Search

Release v0.6.0 - MongoDB Full-Text Search

This release implements native MongoDB full-text search with text indexes, improving search performance by 10-100x for large datasets.

New Features

• MongoDB Full-Text Search
  • Added text index on title, tags, and notes fields with weighted relevance (title: 10, tags: 5, notes: 1)
  • Modified package_search() to use MongoDB $text operator for simple text queries
  • Added relevance-based sorting using $meta: "textScore"
  • Maintained backward compatibility with Solr-style field queries (field:value)
  • 10-100x performance improvement over regex-based search for large datasets
  • Built-in stemming, stop words, and case-insensitive matching

Testing

• Added comprehensive test suite (test_mongodb_fulltext_search.py) with 9 tests
• All 990 tests passing
• CI verified across environments

Docker Images

Available on Docker Hub:

• `rbardaji/ndp-ep-api:0.6.0`
• `rbardaji/ndp-ep-api:latest`

Full Changelog

For detailed changes, see CHANGELOG.md

v0.5.4

What's Changed

Bug Fixes

• fix: normalize group paths in authorization service - Fixed an issue where user groups with leading slashes (e.g., /ndp_ep/ep-123) wouldn't match configured groups without slashes (e.g., ndp_ep/ep-123). The comparison now normalizes both paths by stripping leading/trailing slashes and converting to lowercase.

Docker Images

• rbardaji/ndp-ep-api:0.5.4
• rbardaji/ndp-ep-api:latest

Full Changelog

v0.5.3...v0.5.4

v0.5.3

Changes

• Renamed ENABLE_ORGANIZATION_BASED_ACCESS to ENABLE_GROUP_BASED_ACCESS
• Added GROUP_NAMES environment variable for comma-separated list of allowed groups
• Authorization now checks if user belongs to any group in GROUP_NAMES instead of single organization
• Backward compatibility maintained with function aliases

Migration

Update your .env file:

# Old
ENABLE_ORGANIZATION_BASED_ACCESS=True

# New
ENABLE_GROUP_BASED_ACCESS=True
GROUP_NAMES=admins,developers

v0.5.2

What's New

Added

• Installation script (install.sh) for fresh Ubuntu systems
  • Interactive configuration prompts
  • Automatic Docker installation
  • Environment file generation with overwrite protection
• Infrastructure services reporting to federation metrics
  • jupyterlab_enabled + jupyterlab_url
  • kafka_enabled + kafka_host + kafka_port
  • s3_enabled
  • pre_ckan_enabled
• Docker Compose profiles for optional services
  • mongodb: MongoDB + Mongo Express
  • kafka: Kafka + Zookeeper + Kafka UI
  • s3: MinIO
  • jupyter: JupyterLab
  • pelican: Pelican Federation services
  • frontend: NDP-EP Frontend
  • full: All services

Changed

• Docker Compose now starts only the API by default
• Healthcheck endpoint changed from /status/ to / (status requires auth)

Docker Image

docker pull rbardaji/ndp-ep-api:0.5.2
docker pull rbardaji/ndp-ep-api:latest

v0.5.1

Fixed

• Added authentication requirement to all /status endpoints
• Fixed tests for resource_patch method in base repository
• Fixed SSL verification test to use explicit configuration

Docker

docker pull rbardaji/ndp-ep-api:0.5.1
docker pull rbardaji/ndp-ep-api:latest

v0.5.0

New Features

Resource Management by ID

• GET /resource/{resource_id} - Get a resource directly by its ID without needing the dataset ID
• PATCH /resource/{resource_id} - Update a resource directly by its ID
• DELETE /resource/{resource_id} - Delete a resource directly by its ID

Resource Search

• GET /resources/search - Search for resources across all datasets with filtering options:
  • q - General search query
  • name - Filter by resource name
  • url - Filter by resource URL
  • format - Filter by format (CSV, JSON, S3, kafka, etc.)
  • description - Filter by description
  • limit and offset - Pagination support
  • Returns resources with parent dataset context (dataset_id, dataset_name, dataset_title)

Tests

• Added 13 new tests for resource endpoints (48 total tests passing)

Docker

docker pull rbardaji/ndp-ep-api:0.5.0
docker pull rbardaji/ndp-ep-api:latest

v0.4.1

What's New

Added

• SSL verification toggle for CKAN connections
  • CKAN_VERIFY_SSL environment variable (default: True)
  • PRE_CKAN_VERIFY_SSL environment variable (default: True)
  • Allows disabling SSL certificate verification for self-signed certificates
  • Fixes SSL errors when connecting to CKAN instances with self-signed certs

Changed

• Refactored URL normalization into _normalize_url helper method in ckan_settings

Usage

To disable SSL verification for CKAN with self-signed certificates:

CKAN_VERIFY_SSL=False
PRE_CKAN_VERIFY_SSL=False

Docker

docker pull rbardaji/ndp-ep-api:0.4.1

v0.4.0

What's New in v0.4.0

New Features

• DELETE individual resources: DELETE /dataset/{dataset_id}/resource/{resource_id} removes a single resource while keeping the dataset and other resources intact
• PATCH individual resources: PATCH /dataset/{dataset_id}/resource/{resource_id} allows partial updates to resource fields (name, url, description, format)

Bug Fixes

• ROOT_PATH now properly propagates to Swagger UI requests (fixes #23)

Documentation

• Added link to "Adding New Catalog Backends" documentation in README (fixes #22)

Docker

docker pull rbardaji/ndp-ep-api:0.4.0

🤖 Generated with Claude Code

v0.3.4 - Use purge for permanent deletion

What's Changed

Fixed

• Use purge instead of delete for CKAN datasets and organizations
  • Changed package_delete to use dataset_purge for permanent deletion
  • Changed organization_delete to use organization_purge for permanent deletion
  • CKAN's soft-delete left datasets in database, preventing organization deletion
  • Now datasets and organizations are completely removed, enabling proper cleanup

Docker Image

docker pull rbardaji/ndp-ep-api:0.3.4
docker pull rbardaji/ndp-ep-api:latest

Full Changelog: v0.3.3...v0.3.4

Release v0.3.2 - MongoDB Organization Name Resolution

🎉 Release v0.3.2

Critical Fix: MongoDB Organization Name Resolution

This release adds automatic organization name to UUID resolution in MongoDB backend, ensuring full compatibility with CKAN search patterns.

Docker Image

• Image: rbardaji/ndp-ep-api:0.3.2 and rbardaji/ndp-ep-api:latest
• Base: Python 3.13-slim
• Size: 632MB (26MB smaller than v0.3.1!)
• Digest: sha256:b1f04fad30a293130d6da5fe527511f299a2027f2e2992bfceb859fd05775ef4

🐛 Critical Bug Fix

MongoDB Organization Name Resolution

• MongoDB now automatically resolves organization names to UUIDs in search queries
• Ensures {"owner_org": "services"} works identically in both CKAN and MongoDB backends
• Applied to all search paths: q, fq, and fq_list parameters
• Impact: All existing code using organization names now works seamlessly with MongoDB
• Example: Search by "services" now correctly returns 5 service datasets
• Maintains full backward compatibility - supports both name and UUID searches

📚 New Documentation (2,342+ lines)

• Architecture Diagrams (593 lines): Complete system architecture visualization
• Repository Pattern Architecture (714 lines): Comprehensive guide to the repository pattern implementation
• Repository Quick Reference (440 lines): Developer quick-start guide
• Stack Presentation (514 lines): Interactive HTML presentation of the complete stack

🚀 Existing Features (from v0.3.1)

• Pelican federation integration for distributed data access
• Repository pattern for catalog backend abstraction (CKAN/MongoDB)
• FastAPI-MCP integration for AI agent communication
• Complete docker-compose stack with web interfaces
• 67% test coverage

📦 Complete Stack

• API: FastAPI-based REST API
• Pelican: Federation services (registry, director, origin, cache)
• MinIO: S3-compatible object storage
• MongoDB: NoSQL database for local catalog with organization name support
• CKAN: Optional data catalog backend
• Prometheus: Metrics collection
• Grafana: Metrics visualization

🔧 Technical Details

• MongoDB stores owner_org as UUID internally
• CKAN allows search by both name and UUID
• Solution: Automatic name→UUID lookup in package_search method
• Applied to api/repositories/mongodb_repository.py (3 strategic locations)
• No breaking changes - transparent to existing code

📝 Full Documentation

• CHANGELOG.md - Detailed version history
• Architecture Diagrams - System visualization
• Repository Pattern Architecture - Implementation guide
• Repository Quick Reference - Developer guide
• Stack Presentation - Interactive overview

🐳 Quick Start

# Pull latest version
docker pull rbardaji/ndp-ep-api:0.3.2

# Or use docker-compose
docker-compose up -d

📊 What's Changed

Full Changelog: v0.3.1...v0.3.2