In Part I of this series, I talked about using metadata to define contextual boundaries in Web search. That approach took data germane to the subject (like birth date and location) and used it to define Web spaces for searching.
In Part II, I want to look at using authoritative structures/references to build Web spaces and do Web search. Instead of using data about the subject, we’ll be using authority to guide our search to (hopefully) more useful Web spaces and search results. Authority can be both search guide and search feature and we’ll look at both. But first, the fundamental question:
What is Authority?
I struggled with vocabulary when I first started considering this word. When I thought of authority I thought of this:
… but that’s not what I mean. Instead in the context of Web search and authority I mean two things:
Expertise Authority
Expertise authorities are institutions and groups who are generally acknowledged to offer useful, factual information. The information may be topical or general. Expertise authorities would include the Library of Congress, WorldCat, the archives of an accredited research group, etc.
Access Authority
Access Authority is either access to something normally unavailable to the general public or the authority to offer an option or action which is usually restricted. Certain top-level domains (like .edu, .mil, and .gov) are examples of access authority because only certain groups are allowed to register and use them. (These TLDs are sometimes called “Restricted TLDs.”) Licensing bodies like the FCC have access authority because they license the ability to run a television station in the United States. Your local municipal government exercises access authority when they issue construction permits.
Authority Everywhere, But Rarely Applied
While Google will include and use authoritative data in its own search results, it’s rare that you get the opportunity to use and direct it yourself in an easy way. One of the big exceptions is the site: search syntax with the top-level domains edu, mil, or gov.
One of the most well-known of Google’s search syntax, site: lets you restrict your search to a top-level domain (like edu) or a domain name (ncsu.edu). You can even restrict your search results to a subdomain (wolfpack.ncsu.edu).
When you add site:edu to your query, you’re using authoritative data because only accredited educational institutions are allowed to register and use .edu domains. When you use site:edu, you’re ensuring that every page in your search results comes from the Web space of an accredited institution of higher learning.
That’s really powerful if you think about it! There are only a few top-level domain searches you can do, including site:edu, site:mil, and site:gov, that will bring you results from only one type of Web site. That doesn’t mean everything on those sites will be useful or relevant to your search, but it does mean that with just one addition to your query you can narrow down your search results in a very meaningful way using the authority of the Internet’s infrastructure!
… to a point. As powerful as it is to be able to focus your search results with one application of the site:edu syntax, it only does so much. You’re finding content from higher education institutions in the United States, but you can’t get any more specific than that. You’re not limiting your search to the location of an educational institution, for example. You’re not limiting your results to only those institutions which are private.
But wouldn’t it be cool if you could? Wouldn’t it be great if you could do something like search the Web space of only Catholic universities in New York? Or all the public universities in North Carolina?
Personally, I thought it would be, so I made a Search Gizmo to put some muscle into a site:edu Google search. Not only is it fun to search with, it also shows you how authoritative systems can focus your results even when you don’t have a very specific query. Let’s talk about Super Edu Search. But first, a brief side note about API keys.
A brief side note about API keys: Super Edu Search and another Search Gizmo I’m highlighting in this article, Congressional Social Media Explorer, require API keys to work. API keys are little strings of text that API services use to verify their users. Super Edu Search requires a Data.gov API key and Congressional Social Media Explorer requires a ProPublica API key. Both keys are free and require only registration, and once you have the key all you’ll have to do with it is copy and paste it into a text box. Think of them as an example of access authority!
Using authority to focus your searches: Super Edu Search
I’ve already mentioned the edu top-level domain as an example of access authority, as only accredited institutions of higher learning can use .edu domain names. But who decides whether an institution of higher education is accredited or not?
You might think it’s the US Department of Education, but it isn’t, at least not directly. Instead of accrediting institutions itself, the Department of Education oversees the organizations which do the acreditting. (Perhaps you could argue that the DoE does acreditting once removed.) Furthermore, the Department of Education aggregates information about the colleges and universities around the United States and makes it available via an API at https://collegescorecard.ed.gov/data/documentation/ .
Super Edu Search, available at https://searchgizmos.com/super-edu-search/ , is an authority search on top of an authority search. You’re searching the US Department of Education’s data to find higher education institutions which meet certain criteria, and then you’re using the authority of .edu web space to do a focused Google search. Compared to a regular site:edu search, you can get ridiculously specific!
Super Edu Search allows you to choose the ownership type, religious affiliation, minority/gender emphasis, and location (by state) of the higher education institutions you’re searching. You’re limited to 200 search results but limiting by state will get you under that limit for most states.
Usually when you do a Web search you want to make your query as specific as possible so you get a limited number of results. With Super Edu Search, that strategy is reversed because authority is doing such a good job of narrowing the Web space you’re searching. Let’s talk about a general search like “climate change” and how it looks across four different site:edu searches.
Climate Change and Site:Edu
If I do a Google search for “climate change” site:edu, it looks like this:
The results are as general as you’d expect and rather varied, though I’m surprised to see a single university take the top two spots in the search results.
But what if we used Super Edu Search to change the Web space we were searching without changing the general nature of a search? Let’s find out.
One of the easiest ways to use Super Edu Search is by searching the Web space of all universities in a particular state. What does a search for the term “climate change” look like when your search results are restricted to universities in Hawaii?
Unsurprisingly, results focus on research from and about Hawaii, from rainbows to reefs. Super Edu Search lets you move your focus from creating a specific, detailed query to creating a specific, detailed space to search, allowing you to play with general searches.
Let’s do another one. The Hawaii search focused on location, but what if we did a search for “climate change” across Jewish universities?
In this case, the top results varied, with topics ranging from faith to art to activism. What jumped out to me here is the reference to both activism and activist groups.
You’ll notice that these two results have featured one university’s domain (and sometimes multiple sub-domains) in the search results. That’s because Super Edu Search works by grouping university Web sites into Google search modifiers that look like this:
(site:aamu.edu | site:alasu.edu | site:gadsdenstate.edu | site:drakestate.edu)
Sometimes one domain name overwhelms the other ones at the top of the search results, but you’ll find more as you explore further down from the top.
Sometimes a Super Edu Search result shows more domains up top, like this result for a search of “climate change” across Historically Black Colleges and Universities (HBCUs).
In the case of these search results, sustainability is addressed but so is environmental justice and disproportionate impact.
Every group has its own perspective about a topic. If you’re a farmer in the midwestern United States you’re thinking about climate change differently than a property landlord in coastal North Carolina, who’s thinking about it differently than a local grocery store manager.
Groups form perspectives in response to problems, changes, and even innovations. The distinctive names, keywords, and vocabulary created by groups around topics is a priceless compilation of experience and understanding that you don’t have unless you’re part of that group.
If you aren’t part of a group, you don’t have access to its perspective. If you don’t have access to its perspective, you can’t use its experience and understanding to build a Web query. (This takes me back around to my eternal first search question: how do I ask for what I don’t know?, but more about that in a minute.)
So if you can’t access the expertise of a group to build a search query then you don’t. Instead, you can try to define the group, discover Web spaces that match that group, and see if there’s a way to aggregate those spaces to one searchable area. Super Edu Search does that by using both the Department of Education’s overriding authority data and the authority data of the .edu top-level domain itself.
But as I said a couple of paragraphs ago, my first, essential search question boils down to “how do you ask for what you don’t know?” With Super Edu Search I showed you how you can use educational authority to define groups and then search them with general topics.
Sometimes defining a group isn’t that easy, though. Sometimes the group whose perspective you’re interested in is made of specific people and, unlucky you, you have no idea who they are. Expertise authority to the rescue! Let’s talk about Congressional Social Media Explorer.
Congressional Social Media Explorer
At this writing there are 435 voting members of the US House of Representatives. You’re probably aware of your own representative and some of the noisier national ones, and you might know some of the other reps in your state. And generally that’s all you need to know!
But if you’re doing research that is adjacent to politics or state and federal government, you need more political knowledge than that, and sometimes it can be tedious to find. If you want to know what representatives of Texas circa 2014 have said about cannabis on social media, for example, you might go look them up from a reference resource, create social media sources, and go through them one by one.
OR you can use an expertise authority to find the politicians and then automatically build the searches yourself. That’s what Congressional Social Media Explorer, available at https://searchgizmos.com/congressional-social-media-explorer/ , does.
In this case the authority is ProPublica, which maintains an API of Congressional member data at https://www.propublica.org/datastore/api/propublica-congress-api . Its data extends to everyone who has ever served in Congress, but the CSME goes back to only the 112th congress (2011-2012), since social media was less-developed before that (as was political response to it.)
Since ProPublica has a reference API with all the names I need if I want to research Texas politicians talking about cannabis, I can search for what I do know – I want social media statements about cannabis from politicians who were Texas representatives circa 2014.
(Note that CSME identifies politicians who were representatives at a particular time, but does not put date-boundaries on searches. In other words, if a Congressperson was in office in 2014 and is still maintaining the social media account they were using in 2014, your search will get a list of all statements they’ve made on that account about cannabis until the present day.)
CSME has a template for doing social media searches and uses the authoritative data from ProPublica to fill it in. Since it maintains information on the Senate as well as the House of Representatives, here are the two Texas senators in 2014:
Several one-click searches use Google to find content (or comments) related to each senator across several platforms. (Also note that replies are indexed in addition to politician statements, so don’t look at Google’s page summary and make assumptions about who said what!)
In this case I’m relying on ProPublica’s authoritative data to build somewhat an admittedly usual interface for exploring the comments of politicians on social media. While I’m here I should probably also admit that I found the ProPublica API ripe for extended goofery. Did you ever want to search for US Senators by generation, or percentage of votes missed, or – um – zodiac sign? Feel free to noodle around with Senator Social Slices: https://searchgizmos.com/senator-social-slices/ .
So far in this article I’ve shown you an example of using access authority to build group spaces for Web search, and expertise authority to build searches for politicians across social media. But authority doesn’t have to be the main aspect of a search – sometimes it can just act as guidance. Let’s talk about Pam’s Pin and MegaGladys.
Authority as search feature
Pam’s Pin, at https://searchgizmos.com/pams-pin/ , is one of the first Gizmos I made. It “translates” a street address to a Twitter location search (Twitter uses latitude and longitude for precise location search, which is not user-friendly.)
If you went to Pam’s Pin and just clicked on the button, you’d get the default search, which is to find all tweets which look like they were made in a 1km radius of the exterior shot used for the Brady Bunch house. Buf if you add a little y in the last option before you click on the button, your search results will be limited to only those tweets from verified Twitter accounts – you’ll be using access authority to narrow down your search results.
I am aware that this example of access authority worked much better before Elon Musk came along, and that identity verification on Twitter may mean anything or nothing. So I humbly ask you to imagine Twitter with a consistent identity verification program, and how being able to quickly separate out verified from non-verified user content in conjunction with a location search could be useful. Imagine all the people, living life in peeaaaceeeee… sorry. Happily I don’t have to ask you to imagine anything with my last example, MegaGladys.
MegaGladys, at https://searchgizmos.com/megagladys/ , uses Wikipedia to help surface authoritative information about persons, places, or things. I actually don’t consider Wikipedia itself an example of expert authority. As much as I admire Wikipedia, I don’t give it the automatic trust I would give to the Department of Education or ProPublica. There are way too many pranks and propaganda wars and general shenanigans going on in its pages.
But Wikipedia links to excellent examples of expert authority and that’s what I’m taking advantage of with MegaGladys. If I used MegaGladys to search for Huey “Piano” Smith, a pioneering New Orleans musician who recently passed away, I would actually get a pretty restrained set of results (Mr. Smith has a extensive article at Wikipedia, but MegaGladys is for summarizing and pointing to other resources.)
While I don’t consider Wikipedia itself that authoritative, it does link to resources like WorldCat and the Library of Congress. If I’m getting minimal information from Wikipedia, or information I’m not sure of, I can explore his listings in WorldCat and the Library of Congress, and if I wish I can pursue finding offline resources like popular books.
In the first part of this series I talked about using item metadata to create contextual boundaries for your searches, and in this part we looked at using authoritative data to create Web spaces and provide useful search context.
The next part is going to be a little different – it’s going to be about popularity and how you can use it to make your search better. But I’m not talking about current popularity, I’m talking about recent popularity and popularity over time. I’m talking about …fossilized attention.
