discourse-upload-migrator

Migrates uploads (images, PDFs, and other attachments) from one Discourse instance to another via the Discourse API.

When moving topics or categories between Discourse sites using the built-in discourse export_topics / export_category rails commands, uploaded files are not included. This script downloads each upload from the source site and re-uploads it to the destination. Because Discourse derives filenames from a SHA hash of file contents, re-uploading the same file produces the same path — healing broken links in imported posts automatically.

After running this script, rebake the affected posts in the destination rails console to rewrite absolute source URLs to local ones.

This is useful when moving topics from one Discourse instance to another, as per runbook: https://digitallysovereign.org/t/move-topics-or-categories-between-discourse-instances/742

There is also the original meta documentation topic which also explains how to move topics, albeit less completely: https://meta.discourse.org/t/move-topics-from-one-discourse-instance-to-another/38930

Note: this script was vibe coded. If you notice any issues or have suggestions for improvements, let me know!

---

Requirements

• Python 3.10+
• Access to both Discourse instances
• Admin API key on the destination site
• Source site must be publicly accessible (uploads are downloaded over HTTP)

---

Installation

git clone https://git.digitallysovereign.org/tobias/discourse-upload-migrator.git
cd discourse-upload-migrator
python3 -m venv venv
venv/bin/pip install -r requirements.txt
cp .env.example .env
nano .env   # fill in your values
chmod 600 .env

---

Configuration

Edit .env:

DEST_URL=https://your-discourse-site.example.com
DEST_API_KEY=your_api_key_here
DEST_API_USER=your_admin_username

Get an API key at: https://your-discourse-site.example.com/admin/api/keys

---

Usage

1. Extract upload URLs from the source site

In the rails console on the source server:

Single topic:

topic = Topic.find(TOPIC_ID)
posts = Post.where(topic_id: topic.id)
urls = []
posts.each do |p|
  UploadReference.where(target: p).each do |ref|
    urls << "https://SOURCE_DOMAIN#{ref.upload.url}"
  end
end
puts urls.uniq.join("\n")
File.write("/tmp/urls.txt", urls.uniq.join("\n"))

Whole category:

topic_ids = Topic.where("category_id = CATEGORY_ID").pluck(:id)
posts = Post.where(topic_id: topic_ids)
urls = []
posts.each do |p|
  UploadReference.where(target: p).each do |ref|
    urls << "https://SOURCE_DOMAIN#{ref.upload.url}"
  end
end
puts urls.uniq.join("\n")
File.write("/tmp/urls.txt", urls.uniq.join("\n"))

Copy the file out:

exit && exit
docker cp app:/tmp/urls.txt /root/urls.txt

2. Run the migrator

Dry run first:

venv/bin/python3 discourse_upload_migrator.py \
  --urls-file /root/urls.txt \
  --dry-run

Then for real:

venv/bin/python3 discourse_upload_migrator.py \
  --urls-file /root/urls.txt

3. Rebake posts on the destination

In the rails console on the destination server:

By category:

category_id = 123  # destination category ID
topic_ids = Topic.where(category_id: category_id).pluck(:id)
posts = Post.where(topic_id: topic_ids)
puts "Rebaking #{posts.count} post(s)..."
posts.each { |p| p.rebake!; sleep 0.5 }
puts "Done!"

By single topic:

topic_id = 456  # ← destination topic ID
posts = Post.where(topic_id: topic_id)
puts "Rebaking #{posts.count} post(s)..."
posts.each { |p| p.rebake!; sleep 0.5 }
puts "Done!"

---

Options

Flag	Default	Description
--urls-file	(required)	Path to file with one upload URL per line
--dest-url	from .env	Override destination URL
--dest-api-key	from .env	Override API key
--dest-api-user	from .env	Override API username
--delay	1.0	Seconds between uploads
--dry-run	off	Preview only, no uploads

---

Troubleshooting

Symptom	Fix
ModuleNotFoundError: No module named 'httpx'	Run venv/bin/pip install -r requirements.txt
❌ Missing required config	Check .env exists and has all three values
✗ HTTP 422	API key may lack upload permissions — use a global-scope key
✗ Not found on source	File was deleted on source — re-upload manually
Links still broken after rebake	Check the post raw still contains source URLs — re-run rebake