Search Results

Today’s Wall Street Journal has a long article-debate on privacy.  The strongest pro-privacy is Christopher Soghoian of the Open Society Institute.  He confuses commercial privacy with government privacy:

“The dirty secret of the Web is that the “free” content and services that consumers enjoy come with a hidden price: their own private data. Many of the major online advertising companies are not interested in the data that we knowingly and willingly share. Instead, these parasitic firms covertly track our web-browsing activities, search behavior and geolocation information. Once collected, this mountain of data is analyzed to build digital dossiers on millions of consumers, in some cases identifying us by name, gender, age as well as the medical conditions and political issues we have researched online.”

When asked “Why is that a problem” he replies

“Many of the dangers posed by digital dossiers do not occur regularly, but are incredibly destructive to people’s lives when they do. An unlucky few will be stalked, fired, surveilled, arrested, deported or even tortured, all as a result of the data kept about them by companies and governments. Much more common are the harms of identity theft or public embarrassment. Even when companies follow best practices—and few do—it is impossible to be completely secure.”

Note that “parasitic firms” are collecting the data which is then used for arrest, deportation, and torture.  A bit of a disconnect. Identity theft is a problem, but the risk is decreasing and the costs are almost always low.  Moreover, identity thieves are crooks, not firms.

What is particularly interesting about the article is the survey data reported.  It demonstrates peoples’ confusion about the issues.  92% of the adults surveyed  “Think that there should be a law that requires websites and advertising companies to delete all stored information about an individual” but between 32% and 47% would like websites to provide information of some sort (ads: 32%, discounts: 47%, or news: 40%) “tailored to their interests.”  But of course these numbers are totally inconsistent.  If websites cannot keep any information about an individual, then they cannot provide tailored information since there will be nothing on which to base the tailoring.  The relevant questions are tradeoff questions, but the reported survey does not address these.

Filed under: advertising, privacy

Source.

Filed under: behavioral economics

Here’s the link to the conference, and the program, which covers all angles of social media and the law.  Predictably, my interest here is competition policy — I’ll be on the 10:45 panel discussing those issues along with: Michael Altschul (CTIA), Nicolas Economides (NYU), Adam Thierer (GMU), and Moderator Steve Wildman (MSU).

The link to the webcast is here.  If you’re there, please come and say hello after the panel.

Filed under: antitrust, technology

In my last post, I discussed Edelman & Lockwood’s (E&L’s) attempt to catch search engines in the act of biasing their results—as well as their failure to actually do so.  In this post, I present my own results from replicating their study.  Unlike E&L, I find that Bing is consistently more biased than Google, for reasons discussed further below, although neither engine references its own content as frequently as E&L suggest.

I ran searches for E&L’s original 32 non-random queries using three different search engines—Google, Bing, and Blekko—between June 23 and July 5 of this year.  This replication is useful, as search technology has changed dramatically since E&L recorded their results in August 2010.  Bing now powers Yahoo, and Blekko has had more time to mature and enhance its results.  Blekko serves as a helpful “control” engine in my study, as it is totally independent of Google and Microsoft, and so has no incentive to refer to Google or Microsoft content unless it is actually relevant to users.  In addition, because Blekko’s model is significantly different than Google and Microsoft’s, if results on all three engines agree that specific content is highly relevant to the user query, it lends significant credibility to the notion that the content places well on the merits rather than being attributable to bias or other factors.

How Do Search Engines Rank Their Own Content?

Focusing solely upon the first position, Google refers to its own products or services when no other search engine does in 21.9% of queries; in another 21.9% of queries, both Google and at least one other search engine rival (i.e. Bing or Blekko) refer to the same Google content with their first links.

But restricting focus upon the first position is too narrow.  Assuming that all instances in which Google or Bing rank their own content first and rivals do not amounts to bias would be a mistake; such a restrictive definition would include cases in which all three search engines rank the same content prominently—agreeing that it is highly relevant—although not all in the first position. 

The entire first page of results provides a more informative comparison.  I find that Google and at least one other engine return Google content on the first page of results in 7% of the queries.  Google refers to its own content on the first page of results without agreement from either rival search engine in only 7.9% of the queries.  Meanwhile, Bing and at least one other engine refer to Microsoft content in 3.2% of the queries.  Bing references Microsoft content without agreement from either Google or Blekko in 13.2% of the queries:

This evidence indicates that Google’s ranking of its own content differs significantly from its rivals in only 7.9% of queries, and that when Google ranks its own content prominently it is generally perceived as relevant.  Further, these results suggest that Bing’s organic search results are significantly more biased in favor of Microsoft content than Google’s search results are in favor of Google’s content.

Examining Search Engine “Bias” on Google

The following table presents the percentages of queries for which Google’s ranking of its own content differs significantly from its rivals’ ranking of that same content.

Note that percentages below 50 in this table indicate that rival search engines generally see the referenced Google content as relevant and independently believe that it should be ranked similarly.

So when Google ranks its own content highly, at least one rival engine typically agrees with this ranking; for example, when Google places its own content in its Top 3 results, at least one rival agrees with this ranking in over 70% of queries.  Bing especially agrees with Google’s rankings of Google content within its Top 3 and 5 results, failing to include Google content that Google ranks similarly in only a little more than a third of queries.

Examining Search Engine “Bias” on Bing

Bing refers to Microsoft content in its search results far more frequently than its rivals reference the same Microsoft content.  For example, Bing’s top result references Microsoft content for 5 queries, while neither Google nor Blekko ever rank Microsoft content in the first position:

This table illustrates the significant discrepancies between Bing’s treatment of its own Microsoft content relative to Google and Blekko.  Neither rival engine refers to Microsoft content Bing ranks within its Top 3 results; Google and Blekko do not include any Microsoft content Bing refers to on the first page of results in nearly 80% of queries.

Moreover, Bing frequently ranks Microsoft content highly even when rival engines do not refer to the same content at all in the first page of results.  For example, of the 5 queries for which Bing ranks Microsoft content in its top result, Google refers to only one of these 5 within its first page of results, while Blekko refers to none.  Even when comparing results across each engine’s full page of results, Google and Blekko only agree with Bing’s referral of Microsoft content in 20.4% of queries.

Although there are not enough Bing data to test results in the first position in E&L’s sample, Microsoft content appears as results on the first page of a Bing search about 7 times more often than Microsoft content appears on the first page of rival engines.  Also, Google is much more likely to refer to Microsoft content than Blekko, though both refer to significantly less Microsoft content than Bing.

A Closer Look at Google v. Bing

On E&L’s own terms, Bing results are more biased than Google results; rivals are more likely to agree with Google’s algorithmic assessment (than with Bing’s) that its own content is relevant to user queries.  Bing refers to Microsoft content other engines do not rank at all more often than Google refers its own content without any agreement from rivals.  Figures 1 and 2 display the same data presented above in order to facilitate direct comparisons between Google and Bing.

As Figures 1 and 2 illustrate, Bing search results for these 32 queries are more frequently “biased” in favor of its own content than are Google’s.  The bias is greatest for the Top 1 and Top 3 search results.

My study finds that Bing exhibits far more “bias” than E&L identify in their earlier analysis.  For example, in E&L’s study, Bing does not refer to Microsoft content at all in its Top 1 or Top 3 results; moreover, Bing refers to Microsoft content within its entire first page 11 times, while Google and Yahoo refer to Microsoft content 8 and 9 times, respectively.  Most likely, the significant increase in Bing’s “bias” differential is largely a function of Bing’s introduction of localized and personalized search results and represents serious competitive efforts on Bing’s behalf.

Again, it’s important to stress E&L’s limited and non-random sample, and to emphasize the danger of making strong inferences about the general nature or magnitude of search bias based upon these data alone.  However, the data indicate that Google’s own-content bias is relatively small even in a sample collected precisely to focus upon the queries most likely to generate it.  In fact—as I’ll discuss in my next post—own-content bias occurs even less often in a more representative sample of queries, strongly suggesting that such bias does not raise the competitive concerns attributed to it.

Filed under: antitrust, business, economics, google, Internet search, law and economics, monopolization, technology Tagged: antitrust, Bias, Bing, Blekko, google, microsoft, search, Web search engine, Yahoo

Last week I linked to my new study on “search bias.”  At the time I noted I would have a few blog posts in the coming days discussing the study.  This is the first of those posts.

A lot of the frenzy around Google turns on “search bias,” that is, instances when Google references its own links or its own content (such as Google Maps or YouTube) in its search results pages.  Some search engine critics condemn such references as inherently suspect and almost by their very nature harmful to consumers.  Yet these allegations suffer from several crucial shortcomings.  As I’ve noted (see, e.g., here and here), these naked assertions of discrimination are insufficient to state a cognizable antitrust claim, divorced as they are from consumer welfare analysis.  Indeed, such “discrimination” (some would call it “vertical integration”) has a well-recognized propensity to yield either pro-competitive or competitively neutral outcomes, rather than concrete consumer welfare losses.  Moreover, because search engines exist in an incredibly dynamic environment, marked by constant innovation and fierce competition, we would expect different engines, utilizing different algorithms and appealing to different consumer preferences, to emerge.  So when search engines engage in product differentiation of this sort, there is no reason to be immediately suspicious of these business decisions.

No reason to be immediately suspicious – but there could, conceivably, be a problem.  If there is, we would want to see empirical evidence of it—of both the existence of bias, as well as the consumer harm emanating from it.  But one of the most notable features of this debate is the striking lack of empirical data.  Surprisingly little research has been done in this area, despite frequent assertions that own-content bias is commonly practiced and poses a significant threat to consumers (see, e.g., here).

My paper is an attempt to rectify this.  In the paper, I investigate the available data to determine whether and to what extent own-content bias actually occurs, by analyzing and replicating a study by Ben Edelman and Ben Lockwood (E&L) and conducting my own study of a larger, randomized set of search queries.

In this post I discuss my analysis and critique of E&L; in future posts I’ll present my own replication of their study, as well as the results of my larger study of 1,000 random search queries.  Finally, I’ll analyze whether any of these findings support anticompetitive foreclosure theories or are otherwise sufficient to warrant antitrust intervention.

E&L “investigate . . . [w]hether search engines’ algorithmic results favor their own services, and if so, which search engines do most, to what extent, and in what substantive areas.”  Their approach is to measure the difference in how frequently search engines refer to their own content relative to how often their rivals do so.

One note at the outset:  While this approach provides useful descriptive facts about the differences between how search engines link to their own content, it does little to inform antitrust analysis because Edelman and Lockwood begin with the rather odd claim that competition among differentiated search engines for consumers is a puzzle that creates an air of suspicion around the practice—in fact, they claim that “it is hard to see why results would vary . . . across search engines.”  This assertion, of course, is simply absurd.  Indeed, Danny Sullivan provides a nice critique of this claim:

It’s not hard to see why search engine result differ at all.  Search engines each use their own “algorithm” to cull through the pages they’ve collected from across the web, to decide which pages to rank first . . . . Google has a different algorithm than Bing.  In short, Google will have a different opinion than Bing.  Opinions in the search world, as with the real world, don’t always agree.

Moreover, this assertion completely discounts both the vigorous competitive product differentiation that occurs in nearly all modern product markets as well as the obvious selection effects at work in own-content bias (Google users likely prefer Google content).  This combination detaches E&L’s analysis from the consumer welfare perspective, and thus antitrust policy relevance, despite their claims to the contrary (and the fact that their results actually exhibit very little bias).

Several methodological issues undermine the policy relevance of E&L’s analysis.  First, they hand select 32 search queries and execute searches on Google, Bing, Yahoo, AOL and Ask.  This hand-selected non-random sample of 32 search queries cannot generate reliable inferences regarding the frequency of bias—a critical ingredient to understanding its potential competitive effects.  Indeed, E&L acknowledge their queries are chosen precisely because they are likely to return results including Google content (e.g., email, images, maps, video, etc.).

E&L analyze the top three organic search results for each query on each engine.  They find that 19% of all results across all five search engines refer to content affiliated with one of them.  They focus upon the first three organic results and report that Google refers to its own content in the first (“top”) position about twice as often as Yahoo and Bing refer to Google content in this position.  Additionally, they note that Yahoo is more biased than Google when evaluating the first page rather than only the first organic search result.

E&L also offer a strained attempt to deal with the possibility of competitive product differentiation among search engines.  They examine differences among search engines’ references to their own content by “compar[ing] the frequency with which a search engine links to its own pages, relative to the frequency with which other search engines link to that search engine’s pages.”  However, their evidence undermines claims that Google’s own-content bias is significant and systematic relative to its rivals’.  In fact, almost zero evidence of statistically significant own-content bias by Google emerges.

E&L find, in general, Google is no more likely to refer to its own content than other search engines are to refer to that same content, and across the vast majority of their results, E&L find Google search results are not statistically more likely to refer to Google content than rivals’ search results.

The same data can be examined to test the likelihood that a search engine will refer to content affiliated with a rival search engine.  Rather than exhibiting bias in favor of an engine’s own content, a “biased” search engine might conceivably be less likely to refer to content affiliated with its rivals.  The table below reports the likelihood (in odds ratios) that a search engine’s content appears in a rival engine’s results.

The first two columns of the table demonstrate that both Google and Yahoo content are referred to in the first search result less frequently in rivals’ search results than in their own.  Although Bing does not have enough data for robust analysis of results in the first position in E&L’s original analysis, the next three columns in Table 1 illustrate that all three engines’ (Google, Yahoo, and Bing) content appears less often on the first page of rivals’ search results than on their own search engine.  However, only Yahoo’s results differ significantly from 1.  As between Google and Bing, the results are notably similar.

E&L also make a limited attempt to consider the possibility that favorable placement of a search engine’s own content is a response to user preferences rather than anticompetitive motives.  Using click-through data, they find, unsurprisingly, that the first search result tends to receive the most clicks (72%, on average).  They then identify one search term for which they believe bias plays an important role in driving user traffic.  For the search query “email,” Google ranks its own Gmail first and Yahoo Mail second; however, E&L also find that Gmail receives only 29% of clicks while Yahoo Mail receives 54%.  E&L claim that this finding strongly indicates that Google is engaging in conduct that harms users and undermines their search experience.

However, from a competition analysis perspective, that inference is not sound.  Indeed, the fact that the second-listed Yahoo Mail link received the majority of clicks demonstrates precisely that Yahoo was not competitively foreclosed from access to users.  Taken collectively, E&L are not able to muster evidence of potential competitive foreclosure.

While it’s important to have an evidence-based discussion surrounding search engine results and their competitive implications, it’s also critical to recognize that bias alone is not evidence of competitive harm.  Indeed, any identified bias must be evaluated in the appropriate antitrust economic context of competition and consumers, rather than individual competitors and websites.  E&L’s analysis provides a useful starting point for describing how search engines differ in their referrals to their own content.  But, taken at face value, their results actually demonstrate little or no evidence of bias—let alone that the little bias they do find is causing any consumer harm.

As I’ll discuss in coming posts, evidence gathered since E&L conducted their study further suggests their claims that bias is prevalent, inherently harmful, and sufficient to warrant antitrust intervention are overstated and misguided.

Filed under: antitrust, business, economics, google, Internet search, law and economics, monopolization, technology Tagged: antitrust, Bing, google, search, search bias, Search Engines, search neutrality, Web search engine, Yahoo

Many antitrust law professors are fond of using arson — e.g., a firm burning down the rival’s factory — as the paradigmatic example of exclusionary conduct that might raise rivals’ costs without plausible efficiency justifications.  Here is a modern example with law school hypothetical written all over it involving a Domino’s Pizza manager burning down a competing Papa John’s franchise:

Two managers of a Domino’s Pizza restaurant in Florida were facing arson charges Saturday after allegedly burning down a competing Papa John’s franchise in an attempt to increase sales, the Ocala Star-Banner reported.

The Oct. 20 fire at the Papa John’s in Lake City, Fla. — about 60 miles west of Jacksonville — was ruled to be a case of arson shortly after it was set alight. The blaze completely gutted the business, Lake City Police Department spokesman Capt. John Blanchard.

An investigation resulted in the arrest of 23-year-old Sean Everett Davidson on Thursday and 22-year-old Bryan David Sullivan on Friday. Both were charged with arson. Both men confessed to the crime and were booked into county jail.

Sullivan said he set the fire so the Papa John’s location would go out of business and sales at his Domino’s location would increase, according to police.

HT: a loyal TOTM reader.

 

Filed under: antitrust, exclusionary conduct

Every year around this time—around week 10 of college football season—we are reminded of the inequity of the Bowl Championship Series (BCS) system. Instead of permitting an open playoff system to determine the college football champion, as is done by most other NCAA sports including Division II football since 1973, and more famously, NCAA basketball, the BCS uses a computer algorithm and polls to decide the contestants according to, among other things, regular season performance, the teams’ conferences (BCS-approved or not), and strength of schedule. In particular, six of the ten BCS playoff slots are set aside for teams from BCS conferences.

Read the full piece here.

Summary

Search engines produce immense value by identifying, organizing, and presenting the Internet´s information in response to users´ queries.1 Search engines efficiently provide better and faster answers to users´ questions than alternatives.

Recently, critics have taken issue with the various methods search engines use to identify relevant content and rank search results for users. Google, in particular, has been the subject of much of this criticism on the grounds that its organic search results—those generated algorithmically—favor its own products and services at the expense of those of its rivals. It is widely understood that search engines´ algorithms for ranking various web pages naturally differ. Likewise, there is widespread recognition that competition among search engines is vigorous, and that differentiation between engines´ ranking functions is not only desirable, but a natural byproduct of competition, necessary to survival, and beneficial to consumers.2 Nonetheless, despite widespread recognition of the consumer benefits of such differentiation, complaints from rival search engines have persisted and succeeded in attracting attention from a number of state, federal and international regulatory agencies. Unfortunately, much of this attention has focused on the impact upon individual websites of differences among search engines´ algorithmic methods of identifying and ranking relevant content, rather than analyzing these differences from a conventional consumer?welfare driven antitrust analysis.

For example, many of these complaints ignore the fact that search engine users self?select into different engines or use multiple engines for different types of searches when considering the competitive implications of search rankings.3 Rather than focus upon competition among search engines in how results are identified and presented to users, critics and complainants craft their arguments around alleged search engine “discrimination” or “bias.”4 The complainants must have in mind something other than competitive decisions to rank content that differ from the decisions made by rivals; bias in this sense is both necessary to and inherent within any useful indexing tool. Yet, critics have generally avoided a precise definition of the allegedly troublesome conduct. Indeed, the term “bias” is used colloquially and is frequently invoked in the search engine debate to encompass a wide array of behavior—generally suggesting a latent malignancy within search engine conduct—with some critics citing mere differences in results across engines as evidence of harmful conduct.5

The more useful attempts to define “bias,” however, focus upon differences in organic rankings attributable to the search engine ranking its own content (“owncontent bias”); that is, a sufficient condition for own?content bias is that a search engine ranks its own content more prominently than its rivals do. To be even more precise about the nature of the alleged “own?content bias,” it should be clear that this form of  bias refers exclusively to organic results, i.e., those results the search engine produces algorithmically, as distinguished from the paid advertisements that might appear at the top, bottom, or right?hand side of a search result page.6 Critics at the Senate’s recent hearing on the “Power of Google” were particularly vociferous on this front, accusing Google of having “cooked”7 its algorithm and of “rig[ging] its results, biasing in favor of Google.”8

Competition economists and regulatory agencies are familiar with business arrangements which give rise to concerns of own?content bias.9 Complaints and economic theories of harm assert that a vertically integrated firm (in this case, Google offers search results as well as products like YouTube and Google Maps) might discriminate against its rivals by “foreclosing” them from access to a critical input. Here, the critical input necessary for rivals´ success is alleged to be prominent placement in Google´s search results. The economics of the potential anticompetitive exclusion of rivals involving vertically integrated firms are well understood in antitrust. The conditions that must be satisfied for these concerns to generate real risk to consumers are also well known. Over a century of antitrust jurisprudence, economic study, and enforcement agency practice have produced a well?understood economic analysis of the competitive effects of a vertically integrated firm´s “discrimination” in favor of its own products or services, including widespread recognition that such arrangements generally produce significant benefits for consumers. Modern competition policy recognizes that vertical integration and contractual arrangements are generally procompetitive; it also understands that discrimination of this sort may create the potential for competitive harm under some conditions. Sensible competition policy involving vertical integration and contractual arrangements requires one to be sensitive to the potential consumer welfare?enhancing potential of such vertical integration while also taking seriously the possibility that a firm might successfully harm competition itself (and not merely a rival).

In addition to the failure to distinguish procompetitive conduct from anticompetitive behavior, critics´ allegations of own?content bias suffer deeper conceptual ambiguities. The perceived issue for Google´s rivals is not merely that Google links to a map when responding to search queries, suggesting one might be  relevant for the user; indeed, rival search engines frequently respond to similar user queries with their own or other map products. Rather, critics find problematic that Google responds to user queries with a Google Map. This is a critical distinction because it concedes that rivals´ complaints are not satisfied by the response that consumers are better off with the map; nor do critics pause to consider that perhaps the Google search user prefers the Google Map to rival products.10 Thus, critics brazenly take issue with the relationship between Google and the search result even where they concede Google produces more relevant results for consumers.11 Rather than focusing upon consumers, critics argue that the fact that Google is affiliated with the referred search result is itself prima facie evidence of competitively harmful bias.12 On its face, this argument turns conventional antitrust wisdom on its head. Conduct that harms rivals merely because it attracts consumers from rivals is the essence of competition and the logical core of the maxim that antitrust protects “competition, not competitors.?13

Critics´ failure to account for the potential consumer benefits from ?own?content bias? extends beyond ignoring the fact that users might prefer Google´s products to rivals´. Most critics simply ignore the myriad of procompetitive explanations for vertical integration in the economics literature. This omission by critics, and especially by economist critics, is mystifying given that economists have documented not only a plethora of procompetitive justifications for such integration, but also that such vertical relationships are much more likely to be competitively beneficial or benign than to raise serious threats of foreclosure.14

The critical antitrust question is always whether the underlying conduct creates or maintains monopoly power and thus reduces competition and consumer welfare, or is more likely efficient and procompetitive. To be clear, documenting the mere existence of own?content bias itself does little to answer this question. Bias is not a sufficient condition for competitive harm as a matter of economics because it can increase, decrease, or have no impact at all upon consumer welfare; neither is bias, without more, sufficient to state a cognizable antitrust claim.15

Nonetheless, documenting whether and how much of the alleged bias exists in Google´s and its rivals´ search results can improve our understanding of its competitive implications—that is, whether the evidence of discrimination in favor of one´s own content across search engines is more consistent with anticompetitive foreclosure or with competitive differentiation.

Critically, in order to generate plausible competitive concerns, search bias must, at minimum, be sufficient in magnitude to foreclose rivals from achieving minimum efficient scale (otherwise, if it merely represents effective competition that makes life harder for competitors, it is not an antitrust concern at all). It follows from this necessary condition that not all evidence of ?bias? is relevant to this competitive concern; in particular, Google referring to its own products and services more prominently than its rivals rank those same services has little to do with critics´ complaints unless they implicate general or vertical search.

Despite widespread discussion of search engine bias, virtually no evidence exists indicating that bias abounds—and very little that it exists at all. Edelman & Lockwood recently addressed this dearth of evidence by conducting a small study focused upon own?content bias in 32 search queries. They contend that their results are indicative of systemic and significant bias demanding antitrust intervention.16 The authors define and measure ?bias? as the extent to which a search engine´s ranking of its own content differs from how its rivals rank the same content. This approach provides some useful information concerning differences among search engine rankings. However, the study should not be relied upon to support broad sweeping antitrust policy concerns with Google.

The small sample of search queries provides one reason for caution. Perhaps more importantly, the non?random sample of search queries undermines its utility for addressing the critical antitrust policy questions focusing upon the magnitude of search bias, both generally and as it relates to whether the degree and nature of observed bias satisfies the well?known conditions required for competitive foreclosure. Further, evaluating their evidence at face value, Edelman & Lockwood misinterpret its relevance (Edelman & Lockwood in fact find almost no evidence of bias) and, most problematically, simply assume that own?content bias is inherently suspect from a consumer welfare perspective rather than considering the well?known consumer benefits of vertical integration. Despite these shortcomings, Edelman & Lockwood´s study has received considerable attention, both in the press and from Google´s critics, who cite it as evidence of harmful and anticompetitive search engine behavior.17 In the present analysis, as a starting point, we first “replicate” and analyze Edelman & Lockwood´s earlier study of a small, non?random sample of search queries in the modern search market. We then extend this methodology to a larger random sample of search queries in order to draw more reliable inferences concerning the answers to crucial questions for the competition policy debate surrounding search engine bias, including: (1) what precisely is search engine bias?; (2) what are its  competitive implications?; (3) how common is it?; (4) what explains its existence and relative frequency across search engines?; and, most importantly, (5) does observed search engine bias pose a competitive threat or is it a feature of competition between search engines?

Part I of this paper articulates an antitrust?appropriate framework for analyzing claims of “own?content bias” and delineates its utility and shortcomings as a theory of antitrust harm; it further evaluates Edelman & Lockwood’s study, methodology and analysis using this framework. Part II lays out the methodology employed in our own studies. Part III presents the results of our replication of Edelman & Lockwood and analyzes antitrust implications for the search engine bias debate; Part IV does the same for our larger, random sample of search queries. Part V concludes.