Dark.Fail is a dual stack (clearnet + .onion) website whose stated purpose is “to compile a list of ‘official’ .onion links to serve as an anti phishing resource”.

To aid in their (and everyone’s) ability to confirm if a given .onion mirror is legitimate or not the Dark.Fail website has proposed the “Onion Mirror Guidelines”.

This blog post discusses the OMG and Ablative’s OMG Ansible role that is available on Ansible Galaxy.

Introduction: What are .onion Mirrors and what is the problem?

Due to some issues in how Tor handles .onion service connections many of the larger .onion service operators have opted to resolve the issue by running multiple instances of the Tor daemon each running a different .onion.

The operators then publish these alternate addresses to their visitors hoping that the legitimate visitors can stay one step ahead of the attackers.

Unfortunately this approach lays bare one of the issues with .onion names, and if users can be confused or misled then it’s fair to say that scammers will be quick to try and exploit them.

The Dark.Fail Onion Mirror Guidelines aim to solve this problem by having Webmasters provide users with cryptographically verified lists of valid mirrors;

OMG | Initial Draft

On October 30th 2019 the Dark.Fail twitter account tweeted out the initial announcement of OMG which is copied below;

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Admins,

To reduce the impact of phishing and to ease automatic PGP verification
of mirrors, dark.fail is now defining the Onion Mirror Guidelines. (“OMG”)

Admins that implement this standard show a commitment to user safety
by proving ownership of all URLs associated with their site, and by
committing to regularly prove control of their PGP key.

Sites which do not implement these guidelines by Dec 1, 2019 will be
marked as "unverified" on dark.fail and listed below all other sites.

DarkDotFail

=========

Onion Mirror Guidelines ("OMG")
Version alpha

You must host these text files at all of your .onion URLs:

/pgp.txt - Required - HTTP 200 text/plain
  - A list of all PGP public keys allowed to announce your official mirrors.
  - May contain multiple PGP keys.
  - All keys must be ASCII armored.
  - Do not list a key here unless it is trusted to sign official .onion URLs. 
  - Example: http://darkfailllnkf4vf.onion/pgp.txt

/mirrors.txt - Required - HTTP 200 text/plain
  - PGP SIGNED list of all official mirrors of your site. 
  - Mirrors must be signed by a PGP key which is in /pgp.txt hosted at all of your URLs.
  - Any line in this file which begins with “http://“ or “https://“ 
    is an official mirror of your site.
  - Mirrors must all host the same content. No related forums, no link lists. 
    Place forums, other sites in /related.txt instead.
  - All valid mirrors must only contain a scheme and domain name, no
    ports or paths.
  - /pgp.txt and /mirrors.txt must have the same content on all of your URLs.
  - Text which is not intended to be parsed as an official mirror must 
    be commented out with a “#” as the first character on the line.
  - Example: http://darkfailllnkf4vf.onion/mirrors.txt

/canary.txt - Required - HTTP 200 text/plain
  - PGP SIGNED message MUST be updated every 14 days. 
  - Can be signed by any key specified in /pgp.txt
  - The message must contain the latest Bitcoin block hash and the current 
    date in YYYY-MM-DD format, with string “I am in control of my PGP key.”
    and must also include the string "I will update this canary within 14 days."
  - If you cannot do this you should not be running a darknet market. 
  - Example: http://darkfailllnkf4vf.onion/canary.txt

/related.txt - Optional - HTTP 200 text/plain
  - PGP SIGNED list of all .onion sites related to your site. 
  - This is where you list forums, link lists, related services.
  - Follow the same rules as /mirrors.txt 

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEbf2uZtQ/we7OuH584uRp3H2MPaIFAl243EgACgkQ4uRp3H2M
PaLlRw/9HS6WDRijreXW0cxHEv7l/BDnuIFiLPQmfytKUzcj5IsU+5+MkVi6riVx
YwEvvZyx0u+f5PR8rigORmhIVm7++NBfYy41IoI9bgWSi2EOyaikUT2Hum5Wcz3m
10qwtv/587Exd0KN1buxzjfGzeLo7h2CgCowR6msQTugx/uzFkmI0qTRpMQP19gC
dbDPpfyK9HFNhEjWQhyWqUPE1qsD3EdUxUItOf+/VG5JZRsLy6/913Oc965cpAAC
dUjldqlksekXrsSrAKmMsy/ZZzPsJIo8mghJLCuoiPSTj2jBoMovRlxNSHBS+w7v
dnhalZh5E/ExCqMMrnwzJJhA5HnelVCmsNmwfXG6KhhhPhEfHETFyPXfBIjJD4wI
28QAaMgiozBd957gdzYUzaemk71tLI/5XuhR4HqakGfGTKio6cb0Mg+KMkEg4gN1
/nsMlIrYvbLV+pfzoveAUn2C20FYhgZR5oJtew36QDqLTRHeEEHnoBnIqGxqh6gY
+fsIybRANl+wi1Pru5FV2/wKzBO6hshyLpWQETmFCqycQLbjOO8qP2bdvpfgPr1f
3Al2saq4R7Cm+VgxKt9C6IA/xsXChawdgTq67AfHNFq2TapXJdAXIvDVfL5S6E4y
Y5CDH1EsBn5pPEWWzmohcTRKaf1zs/SOXnkLGV8JezuKXFzFi1U=
=LcSi
-----END PGP SIGNATURE-----

RFC 8615 | Ablative’s Proposed Improvement

Lot’s of initiatives (such as https://securitytxt.org/ ) comply with RFC 8615 which is known as “Well-Known Uniform Resource Identifiers (URIs)”. This defines a standard where these files are either in /.well-known/ or in a sub-directory there-of.

Because pgp.txt, mirrors.txt and related.txt are ambiguous I suggested to the Dark.Fail team that OMG moves to RFC 8615 with a sub-directory of /.well-known/omg/.

Trusting that they’d accommodate this request I went ahead with writing the Ansible role.

The Ansible Role

The role is quite simple and only has a handful of defaults;

---
gpg_key_id: "gpg@example.onion"
canary_line_one: An automated bot has confirmed this canary
canary_line_two: An automated bot has control of this PGP key
canary_days: 14
temp_dir: /tmp
remote_dir: /var/www/htdocs
mirrors_list: []

You’d need to override gpg_key_id and will likely want to change canary_line_one and canary_line_two to something a bit more fitting to your situation.

mirrors_list should match across all of your hosts and remote_dir defaults to OpenBSD’s htdocs directory, this’ll likely need to be overridden on a per-host or per-platform basis depending on your needs.

The first set of tasks is ensure that we have an RFC 8615 compliant directory structure on the host and then uploads the pgp.txt file to .well-known/omg sourced from a local omg_pgp.txt;

- name: Create the remote .well-known folder
  file:
    path: "{{ remote_dir }}/.well-known"
    state: directory
    mode: '0755'

- name: Create the remote .well-known/omg subfolder
  file:
    path: "{{ remote_dir }}/.well-known/omg"
    state: directory
    mode: '0755'


- name: Copy the the unsigned list of pgp keys from files/omg_pgp.txt to the hosts
  copy:
    src: files/omg_pgp.txt
    dest: "{{ remote_dir }}/.well-known/omg/pgp.txt"
    mode: '0644'

Ansible will search in various places for files which is why the source is named omg_pgp.txt.

With the pgp.txt file uploaded Ansible writes out the mirrors.txt list to a local temporary location and signs it. Note the use of run_once: true on some of these tasks;

- name: Write out the mirrors list using the mirrors_list variable to a temporary location
  template:
    src: mirrors.txt.j2
    dest: "{{ temp_dir }}/mirrors.txt"
  delegate_to: localhost
  run_once: true

- name: Sign the mirrors file
  command: "/usr/bin/gpg2 --clear-sign --local-user {{ gpg_key_id }} {{ temp_dir }}/mirrors.txt" # noqa 301
  register: mirrors_signed
  delegate_to: localhost
  run_once: true

- name: Copy the signed mirrors file to the hosts
  copy:
    src: "{{ temp_dir }}/mirrors.txt.asc"
    dest: "{{ remote_dir }}/.well-known/omg/mirrors.txt"
    mode: '0644'

Finally we create the canary.txt, this requires us to fetch and parse a remote JSON document containing the Bitcoin blockchain hash;

- name: Get the blockchain hash
  get_url:
    url: "https://blockchain.info/latestblock"
    dest: "{{ temp_dir }}/blockchain_info.json"
  delegate_to: localhost
  run_once: true

- name: Parse the blockchain data
  include_vars:
    file: "{{ temp_dir }}/blockchain_info.json"
    name: blockchain
  delegate_to: localhost
  run_once: true

- name: Write out the canary to a temporary location
  template:
    src: canary.txt.j2
    dest: "{{ temp_dir }}/canary.txt"
  delegate_to: localhost
  run_once: true

- name: Sign the mirrors file
  command: "/usr/bin/gpg2 --clear-sign --local-user {{ gpg_key_id }} {{ temp_dir }}/canary.txt" # noqa 301
  register: canary_signed
  delegate_to: localhost
  run_once: true

- name: Copy the signed canary file to the hosts
  copy:
    src: "{{ temp_dir }}/canary.txt.asc"
    dest: "{{ remote_dir }}/.well-known/omg/canary.txt"
    mode: '0644'

And that’s all there is to it, depending on how you secure your GPG keys will dictate whether this runs on a schedule or whether it will require manual intervention every time. Thankfully with the help of this playbook even manual runs will be trivially easy.

asciicast