arXiv Connector

What this page helps you do

Use this connector to ingest external content into your RetainDB project for retrieval and memory grounding.

Prerequisites

• A project id.
• Connector-specific credentials/config.
• API key or dashboard access.

Request and response

Create source

{
  "name": "arXiv Connector",
  "connectorType": "arxiv",
  "config": {
    "query":"retrieval augmented generation","max_results":50
  }
}

Trigger sync

curl -X POST "https://api.retaindb.com/v1/sources/src_123/sync" \
  -H "Authorization: Bearer $RETAINDB_API_KEY"

Step-by-Step Workflow

1. Create source under /v1/projects/:projectId/sources.
2. Trigger /v1/sources/:sourceId/sync.
3. Poll /v1/sync-jobs/:jobId.
4. Validate retrieval against freshly synced material.

Try it

curl -X POST "https://api.retaindb.com/v1/projects/proj_123/sources" \
  -H "Authorization: Bearer $RETAINDB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name":"arXiv Connector",
    "connectorType":"arxiv",
    "config":{"query":"retrieval augmented generation","max_results":50}
  }'

What to expect

• Source creation is immediate.
• Sync latency depends on source size and provider limits.
• Retrieval consistency improves after sync reaches terminal success.

Troubleshooting

• Too-broad query: tighten keywords and date windows.
• Low relevance: increase rerank quality profile.
• Sync job stuck/failed: inspect /v1/sync-jobs/:jobId and retry.

Security notes

• Store connector credentials in secure secrets.
• Keep source/project ownership tenant-scoped.

Verify your setup

• Source listed under project.
• Sync job completes.
• Synced content appears in search results.

Next step

• Projects and sources API
• Dashboard source workflow