Over 15 years ago I wrote a book called Information Trapping. It was about how to set up online monitors to find online information around certain keywords and keep it coming as a flow to you via tools like RSS, page monitors, etc.

As you might imagine, Information Trapping’s resources and tool listings are very out of date now. Still, the skill of monitoring information online is critical to what I do. I can’t write a twice-daily Internet newsletter by myself without being able to trust that an ongoing current of useful, credible information makes it to my email and RSS feed reader. In years past, that has meant tools like Undrip and Nuzzel and Google Alerts.

Undrip died in 2013, Nuzzel was acquired by Twitter and shut down in 2021, and Google Alerts has started sending me articles over ten years old as breaking news. Because of that, and because Internet monitoring services are usually not price-friendly to independent researchers anyway, I’ve been spending more time making my own solutions, like Calishat Snaps. Most of the things I’ve been making require non-free APIs, so I can’t share, unfortunately. Over the weekend, however, I made a tool for monitoring Mastodon hashtags and emailing a daily digest of the latest posts, and that I can make available for free. You don’t even need to have a Mastodon account to use it; all you need is Google Sheets and an email address. Let me show you the Mastodon Hashtag Monitor. But first, a little about Mastodon if you’re unfamiliar.

What’s Mastodon?

Mastodon is a social media platform. It’s set up a lot like Twitter with a focus on short posts. Mastodon’s been around for a good while but has come into more prominence in the last couple of years as Twitter continues its transformation into a hellscape. Unlike Twitter and other social media platforms like Facebook, Mastodon is decentralized. With Twitter and Facebook, all users go to the same Web site (like Facebook.com) to log in and participate. With Mastodon, users can log on to different Web sites (called “Instances”) that are part of Mastodon’s network and, via the network, communicate with people on other Mastodon Instances. Thus, Mastodon is not one giant social media Web site, it’s lots and lots of smaller sites networked together and communicating. (If you want a more in-depth overview of Mastodon and how it works technically and culturally, I recommend the excellent writeup at Notes from a Data Witch.)

Because it’s decentralized, the question of how to search Mastodon is more complicated than it is for a centralized social media platform like Twitter. If you want to search Twitter, you go to Twitter and type your query in a search box. If you want to search Mastodon, you either have to search individual Instances — single Mastodon Web sites — one at a time, or you have to use a search tool designed to search several Mastodon instances at once. (I made one if you’d like to try it. It’s called Big Mastodon Hashtag Search and it’s free to use and has no ads. It searches 14 Mastodon Instances by default and gives you the option to add your own to the list.)

The conventional wisdom I’ve seen around the social media world is that Mastodon isn’t good for searching and monitoring because it’s decentralized and you can’t get enough results to make it worthwhile. I think that’s a pile of road apples because it equivalates the value of all searchable content.

I mean, if you can stand it go to Twitter and do a search for anything the slightest bit popular. Look at the results you get. You’ve got some garbage in there, don’t you? You might have A LOT of garbage depending on what you searched for. Does it help your search, that you got ALL the garbage? Does it help your information-aggregating task that your search space was centralized such that you automatically got every bit of junk from every nook and cranny? My guess is no!

Contrast that to Mastodon. There’s a large, well-known Mastodon instance and literally thousands of smaller ones. Unlike a centralized social media platform, with Mastodon it’s up to you to create your own search space by aggregating a list of Instances to search. (I’ll show you how to do that a little later in the article.) Of course, you can restrict your search to the Mastodon’s largest Instance, Mastodon.social, and get plenty of results, but it’s much more to your benefit to take the time to aggregate a list of useful Instances to search.

And yes, doing this takes longer than it does to just plug a search term into Facebook! But it’s worth it. By choosing a list of Mastodon Instances you want to monitor, you are shaping your own search space. Done properly this pays off in richer search results and less garbage to wade through.

Consider other possibilities as well. What if every EU member had its own Mastodon instance that it used to communicate official announcements? Imagine those Instances were aggregated into one search space. You could monitor for general hashtags like #news and #resources because your search would be limited to a completely credible set of instances; you wouldn’t have to worry about bots and scammers bombing your hashtags with garbage. Sounds good, doesn’t it? Decentralized social media means that we, the searchers, have much more power in creating our searching spaces and we should be very excited about that. (So should credible information sources!)

Let’s try a practical application of aggregating Mastodon search spaces by setting up the Mastodon Hashtag Monitor.

Using the Mastodon Hashtag Monitor

You do not have to have a Mastodon account to use the MHM. You will have to have a Google Drive account, because the MHM is based on a Google Sheet instead of a Web site. I created it as a Google Sheet so Mastodon Hashtag Monitor could use Google’s task scheduling and email tools instead of me having to find and integrate my own solutions. The first thing you’ll need to do is get your own copy of the sheet.

Getting A Copy of the MHM

To get a copy of the MHM, go to the Google Sheet link here. It’ll look like this:

See how it’s set to “View only”? You have to make your own copy, which you’ll customize with your own information. Choose File -> Make a Copy. If you’ve made copies of Google Drive documents before, you might notice that the popup you get looks a little different:

In addition to letting you set the name of your copy, Google Sheets is also warning you that there’s a Google Apps Script file, Mastodon Alerts, attached to the file. Before you’re able to use Mastodon Hashtag Monitor, you’ll have to give the Google Apps Script permission to run. We’ll look at that a little later in the article, but for now let’s set up what you want to monitor.

Customizing MHM

Once you’ve created your own copy of MHM it should look like this:

The gray cells in column B are where you specify what you want to monitor.

Hashtags to Monitor

Put the hashtags you want to monitor for here, separated by commas. Do not use the hashtag sign (#). It’s set up by default to monitor for the hashtags GiftArticle and GiftLink.

If you’ve blanking on what hashtags to monitor, I made a Web tool you might find useful; it’s called Hashtag Harvest. Enter a hashtag and HH searches four large Mastodon Instances and returns the most popular related hashtags with links to explore them. (It also lists Mastodon users who have affiliations with a hashtag, in case you have a Mastodon account and you’re looking for community.) Hashtag Harvest is free to use and has no ads.

Instances

After you’ve specified the hashtags you want to monitor, you’ll need to specify the instances you want to monitor on. There are a couple of large Instances listed here by default; if you’re just experimenting or using general keywords that may be all you need. But if you’re more interested in keywords that have a specific focus, be it topical or geographical, it’s worth your while to explore what other relevant Instances might be available.

(Please note that when entering Instances, you should enter the domain name only! Do not preface the instance name with https:// — if you do it won’t work! )

If you’re just getting started monitoring Mastodon and you want to add a bunch of large instances to your monitoring list and see what you get, Instances.Social might come in handy. It offers both a wizard for finding Mastodon Instances and a list of Instances in a few different formats. If you want to do something like find all Instances which use English and have over 50,000 users, that’s no problem.

Finding instances by keyword search isn’t as easy; I wasn’t able to find a Instances.social keyword search I liked, so I made my own. It searches both title and description of an Instance for a keyword. This is a great way to find smaller, more focused communities.

The Instances.Social keyword search tool is available on my GitHub: https://github.com/ResearchBuzz/mastodon-instance-keyword-search . You’ll need your own Instances.Social API token to use it (tokens are free and you can get them here.) Download the HTML file, open it in a text editor, paste your API token where instructed, save the HTML file, open it in your favorite browser, and you’re good to go.

Excluding Hashtags and User Names

When I was testing the MHM, I realized I needed a mechanism to screen out user names and hashtags. Mastodon is mostly pretty good about not having spam, but when you’re monitoring Mastodon.social, which at this writing is the largest instance, you will occasionally get people who are not quite breaking the rules with their posts, but whose content really isn’t helping your search results. And of course, sometimes you might want news with a #politics hashtag, but you want to filter out posts that also contain the hashtag #business because they’re not useful to your topic.

Enter a list of hashtags separated by commas to exclude them. To exclude usernames, enter usernames separated by commas. Be sure to enter only the user name and not the entire address; if you want to exclude SpammyPoster@mastodon.example.com, enter only SpammyPoster in the Users to Exclude part of the form.

EMail Address

Finally, enter the address where you want your daily reports to go. Right now MHM supports sending a report to only one e-mail address.

Now that you’ve established what hashtags you want to search and which Instances you want to use to search them, you’ll need to grant some permissions to the Google Apps Script that makes everything go. To access the script, go to Extensions -> Apps Script. You’ll see a screen which should look like this:

MHM’s Google Apps Script

The first thing you have to do is make sure that the script is set to run the right function. In the screenshot there’s an area that reads formatDate. If you click on the triangle beside that, you’ll get a list of possible functions to run.

That area should be set to main, so that the area looks like this:

The function to run must be set as main for the program to work.

Once you’ve made sure you’ve set the script to run the main function when executed, you need to grant permission for the program to run. This is where Google tries to scare you, as it should. (The good news is you only have to set permissions once.)

Setting MHM’s Script Permissions

Click on the Run button to start the program. You’ll almost immediately get a popup that you need to grant the script access permissions.

Click on Review permissions.

You’ll get a warning that you’re about to do something potentially unsafe because of an unknown developer (your email address will be noted as the developer’s, because you made a copy of the sheet. I have blacked out my test account address.) Click on Go to Mastodon Alerts (unsafe). Google will then give you a very scary looking warning:

Here’s why Google is warning you that those permissions are necessary:

See/edit/create Google Sheets data – The program uses the information you put on the Google Sheet to run the Mastodon searches.

Connect to an external service – The program is making Mastodon API calls to get hashtag information.

Send email as you – You’re authorizing the program to send email from a Google Sheet you own.

While your granting these Google permissions means that Mastodon Hashtag Monitor could in theory do all kinds of crazy things, if you review the script you’ll see that those permissions are being used only in the service of helping you monitor Mastodon. Click on Allow to continue.

Testing the Script

Now that you’ve granted MHM the necessary script permissions to run, it’s time to test it.

Mastodon Hashtag Monitor was designed to be run daily. Because of that it filters out all search results more than 30 hours old. If you’re using an obscure or niche keyword you might find that your test doesn’t get any results that recent. Therefore I recommend that you include giftarticle and giftlink for your test run; those hashtags have results most of the time. You can take them out if the test works.

Testing the script is as easy as clicking the run button again. Since you’ve granted permissions to the script, you’ll see a bunch of things happening in the Execution Log.

And a moment after that you should get an email!

The email starts with the three most replied to / reposted / favorite posts for your hashtag and beneath that includes a list of the most frequently-appearing hashtags and most active users for your hashtag query:

And finally, it lists the posts containing your hashtag over the last 30 hours:

Everything looks great! The last thing you need to do is schedule the script to run every day so you can start your morning off with some Mastodon hashtag goodness. Head back to the Google Apps Script page (it’s Extensions -> Apps Script from your Google Sheet) and choose the little alarm clock on the left menu. That takes you to the Triggers setup.

Click on the blue “Add Trigger” button on the lower right to open the Add Trigger for Mastodon Alerts window. Yours won’t look exactly like this but as you adjust the preferences you’ll see it change.

Function to run – Must be main, otherwise the program won’t work.

Choose which deployment should run – Leave on Head.

Select event source – Time-driven.

Select type of time based trigger – Day timer.

Select time of day – This is up to you. I generally prefer to get my updates early in the morning. Since the program retains the last 30 hours’ worth of posts, I recommend you choose a time of day which is less posting-active, such as very early in the morning (as in this example) or very late at night.

And don’t miss that Failure Notification Setting! If there’s a problem with the script you’ll get an email when it runs. Once you’ve chosen a time you like click Save. If it’s worked, your Triggers page will update with a new entry.

That’s it, you’re all set! If you want to change the hashtags and Instances you’re monitoring, all you have to do is update the spreadsheet’s form. Don’t want to monitor Mastodon hashtags anymore at all? Just delete the spreadsheet!

Future Considerations

I have some vague ideas about things I want to do for future versions of this tool, but I want to get a couple of weeks’ worth of results back to see what I find most irritating and useful. I’m thinking about maybe excluding hashtags by instance level and incorporating some full-text filtering to the found results. Not sure yet. Stay tuned, I appreciate you coming along for the ride.