This post was last updated on July 11th, 2019 at 07:59 pm
On March 19, 2019, Bitwise, a crypto asset management firm, filed a report with the U.S. Securities & Exchange Commission (SEC) in hopes of getting approval for their Bitcoin ETF.
In the report, Bitwise asserted that 95% of exchange volume was fake and only 10 exchanges had “real volume.”
This report spread like wildfire throughout crypto Twitter.
Today, I’m joined by Nathaniel Whittemore, curator of Long Reads Sunday and coauthor of the 6,000-word mammoth on our blog entitled, “Crypto Market Cap: An In-Depth Review & Survey Of Emerging Alternatives.” In this episode, we turn the tables as he interviews me about my take on the Bitwise report.
In this episode, I’m also excited to share with you the launch of our brand new Exchange Transparency Ratings. The purpose of these ratings is to rank cryptocurrency exchanges by their willingness to provide auditable history.
Please enjoy this conversation between myself and Nathaniel Whittemore.
Topics Discussed In This Episode
- The proper context for understanding the purpose of the Bitwise report
- A high-level overview of what was included in the report
- How the analysis is fair for a Bitcoin ETF, but probably not fair for considering the rest of the cryptosphere
- Why clearly defining terms and methodology matters
- A primer on crypto market data
- What the best exchanges have in common
- What the worst exchanges lack
- What crypto Twitter got wrong about the report
- How inaccurate crypto data is negatively impacting our everyday lives
- Why we must demand for accurate data from exchanges
- How the Nomics API is addressing the data problem
- The reasons why accurate data benefits everyone, including exchanges
- Long Reads Sunday
- Nathaniel Whittemore
- Nathaniel Whittemore on Twitter
- Bitwise Asset Management
- Bitwise on LinkedIn
- Bitwise on Twitter
- Bitwise Bitcoin ETF Report
- Securities & Exchange Commission (SEC)
- Coin Market Cap
- Nomics Pricing Methodology
“This report came out and crypto Twitter did not question this. From a space that tends to be incredibly critical and skeptical of a lot of things, I was surprised how much scrutiny this report did not get from crypto Twitter.”
“Our approach with Nomics is to say, ‘Hey can we do things that incentive exchanges to provide better data?’”
“What I would call for is for users and for the industry to tell exchanges that they want transparency, that they want history, and individual trade level data.”
Welcome to Flippening, the first and original podcast for full time, professional, and institutional crypto investors. I’m your host, Clay Collins. Each week we discuss the cryptocurrency economy, new investment strategies for maximizing returns, and stories from the frontlines of financial disruptions. Go to flippening.com to join our newsletter for cryptocurrency investors and find out just why this podcast is called Flippening.
Clay Collins is the CEO of Nomics. All opinions expressed by Clay and podcast guests are solely their own opinion and [00:00:30] do not reflect the opinion of Nomics or any other company. This podcast is for informational and entertainment purposes only and should not be relied upon as the basis for investment decisions.
Clay: My guest today is actually me. In this episode, I’m joined by Nathaniel Whittemore, crypto thought leader and creator of Long Reads Sunday on Twitter. Nathaniel is going to be interviewing me.
Here’s the backstory around the discussion that Nathaniel and I are about to have. On March 19th, Bitwise came out [00:01:00] with a report that made two claims that were of particular interest to us at Nomics. The first claim was that there were only 10 exchanges that had actual volume. The second claim was that 95% of reported exchange volume is fake.
As a data company, the first thing that we did was look at these exchanges and see what commonalities were found among both the good actors and the bad actors. Here’s what we found. Eight of the 10 exchanges that Bitwise identified as good actors provide some [00:01:30] of the most granular data in the space about exchange activity and they provide this data with full and complete history. That is the main thing that we found in common among the trusted exchanges identified by Bitwise is that they were very transparent about trading activity.
In contrast, we found that of the exchanges that Bitwise explicitly calls out as bad actors, every single one of them provide limited trading history and virtually no granularity around trading activity. In other words, the good actors from Bitwise’s [00:02:00] report had highly transparent data practices and the bad actors were not transparent.
When you think about it, it makes sense that opacity around exchange data would be correlated with fake volume, toxic activity, and wash trading. Indeed, just like an IRS audit, the more data history and granularity provided by an exchange engaging in nefarious activities, the more likely they are to be caught so they have no motivation to be transparent.
On the other hand, upstanding exchange operators have every incentive to [00:02:30] provide highly granular data with as much history as possible as this kind of transparency attracts market makers, generates the broader discovery of an exchange’s markets and trading peers, and engenders trust among institutional traders, investors, and regulators. On the basis of this analysis, we at Nomics have decided to announce our exchange transparency rating system. At the time that my conversation with Nathaniel was recorded, our transparency and rating system for exchanges wasn’t announced and wasn’t live, [00:03:00] so during this interview, Nathaniel and I are talking as if we hadn’t launched our rating system.
Anyway, you can find our ratings by going to nomicsratings.com and I should also mention that we’re going to make these ratings available via our API in addition to aggregate data filtered by these ratings. So, suffice to say, we’re excited about the Bitwise report and the prospect of a Bitwise ETF. Their report does a lot to educate the space. It’s directionally correct and has forced a lot of people in our industry [00:03:30] to do much, much better.
But, we also believe that the Bitwise report left a lot of questions unanswered. To be honest, I’ve been surprised by two things about the report. First, I’ve been struck by the lack of scrutiny around the report particularly by crypto Twitter, which has the reputation for being an all-out war zone. Second, I’ve been surprised by the degree to which folks took a document meant to market the approval of Bitwise’s ETF to the SEC and over-generalized the findings [00:04:00] to apply to all exchanges, even ones not analyzed by Bitwise and also to overgeneralize the findings to all tokens even though, again, Bitwise didn’t look at all tokens, as far as we can tell, in the report.
In short, I think that the crypto communities response to the report lacked criticality and nuance. Don’t get me wrong, as a Bitcoin hodler, I want this ETF to be approved and of everyone who submitted an application, I want Bitwise’s ETF to be accepted first [00:04:30] because I think that they know the space better than other applicants. But, I believe that the claims made by the report are just too large and global to take at face value and that a more nuanced conversation needs to be had.
With all of this established, I’d like to spell out our seven critiques of the Bitwise report. Critique number one is that the report’s primary purpose is to persuade the SEC to allow a Bitwise ETF. For this reason, it has an inherent bias, that is the document is a marketing document first and a research [00:05:00] document second. As a research document, it was not subject to the formal peer review system that academic research papers are typically subject to.
Critique number two is our most superficial criticism and that is that the report seems to have a base-10 bias. The number 10 seems very curious. Why are there only 10 good exchanges not nine or 11? Critique number three is that their stated findings are not trustless, which means that they are not falsifiable. That is when the [00:05:30] Bitwise team says that only 10 exchanges have actual volume, the determination about which exchanges pass muster versus not appears to be a 100% qualitative call. That is when the Bitwise team says that only 10 exchanges have actual volume, the determination about which exchanges pass muster versus not are made by people.
Someone is looking at the data and giving individual exchanges a thumbs up or a thumbs down and there is no stated, [00:06:00] independent test that stands apart from Bitwise that a third-party can independently apply. You either have to trust Bitwise’s conclusions or not and this leads me to critique number four which is that the report’s conclusions are stuck in time. That is because of a lack of formal methodology that can be independently used to rate an exchange, a third-party cannot update the list. So, today, some of the exchanges identified by Bitwise as being good actors might have flipped and be engaging [00:06:30] in wash trading and fake volume practices and exchanges that Bitwise did not indicate have actual volume may have cleaned up their practices.
Critique number five is that the report only displays analysis on BTC markets, primarily BTC to USD and BTC to Tether pairs. Therefore, while Bitwise’s analysis was perfect for a Bitcoin ETF, and we don’t fault them for only doing Bitcoin analysis. It’s a mistake to apply their findings broadly to all crypto markets. [00:07:00] Critique number six is that there is no stated time frame for some of the most important data points in the report. For example, they assert that 95% of volume on global crypto exchanges is fake. Is 95% of the volume fake for 2019 year to date? Is this for all of 2018? Is it for a specific date?
Our final critique of the report is that it is unfair to upstanding exchanges not included in the Bitwise list. There are many exchanges like ShapeShift and IDEX that have gone out of their way to comply [00:07:30] with regulators. In ShapeShift’s case, they’ve ostensibly experienced layoffs as a result of this compliance.
Anyway, this monologue has gone on long enough. Let’s go without further ado to my conversation with Nathaniel Whittemore from Long Reads Sunday.
Nathaniel: I’m Nathaniel and I’m @nlw on Twitter and I do this thing [00:08:00] every week called Long Reads Sunday. The idea of Long Reads Sunday is basically that I’m spending a huge amount of time trying to ingest, not only the news, but people’s reactions to the news and trying to understand what’s happening in crypto on a level that’s more than just the facts but is also the interpretation. When something gets the whole community talking, my first instinct is to ask someone who knows more than I do about the topic to get a read on how to interpret it.
Over the last couple weeks, there’s been a huge amount of conversation [00:08:30] around exchange data and are volumes real and qualities of data. It’s led to companies introducing new products or new features. It’s had a lot of people talking. I think, for me, the interesting thing is exchange data practices have been this open source of discontent but not so much that we’re actually willing to do something about it or change our habits yet, at least on the [00:09:00] surface. I don’t think that’s actually the case in total, but it’s interesting to me when something goes from yeah, it’s part of the industry, it’s just part of the Wild West to the narrative shifting to, this is something to be changed, this is something that we need to address and here’s how we do it.
I think that’s part of the interesting thing about this Bitwise report is that it came with its own followup. Anyways, I think what would be great, Clay, is maybe if we can just dive in by [00:09:30] you giving a little bit of context about what was actually released, what it said. Then from there, what I want to get into, what I found so interesting about your response was almost more like this is a moment for the 301 level education around exchange data and what we can do to be better about it, basically. That’s where I’d like to get, but maybe let’s start with what actually happened with the Bitwise report.
Clay: Just at a high level, Bitwise is [00:10:00] trying to make a go of getting an ETF approved. So, they put out a doc which they submitted to the SEC marketing in favor of their ETF. This is a really smart thing to do. I think before we get into the report itself, I think it’s important to comment on what this report is. So, this report is in part [00:10:30] something that you know a lot about, Nathaniel, it is a marketing document. It is there to create a narrative.
Again, that’s smart of Bitwise to do. There’s nothing wrong with marketing as long as it’s truthful, but that is the context. I guess digging a bit into this a little bit deeper, I think it’s worth noting that historically, Bitcoin ETFs have been rejected because of concerns about [00:11:00] trading activities of the asset which would be underlying the asset, namely Bitcoin, on exchanges. Bitwise needed to establish that they had found a way to address the number one concern that had led to the rejection of past ETFs, which is around exchanges, trading activity, and exchange volume, which is one of the reasons why [00:11:30] they focused on this issue the to extent that they did.
Nathaniel: Got it, yeah. So, I think just quickly before we dig into it a little bit more on the specifics, I’ve actually literally written the line that the importance of recognizing that narratives are marketing is not to diminish them, but to create a bulwark against reactivity. I think it’s really important to recognize that recognizing something as marketing doesn’t delegitimate it. [00:12:00] It just puts it in context, right? It allows us to understand what to do with it.
I think that one of the things that’s really interesting watching the narrative around this too is that the easy soundbite was 95% of volume was fake or something like that. But in point of fact, what they were actually arguing as about the health of the ecosystem and their point was trying to peel back layers to actually allow people to see what was going on on a more granular level. I think it’s already [00:12:30] more complex than what would have you say. I guess that let’s talk about the conclusion that they came to ’cause I think that they looked at 81 exchanges and ended up arguing that 10 passed their test, their heuristic of real volume.
Clay: What they came out and said was that 10 of the 81 top exchanges that they looked at were revealed [00:13:00] to have actual volume. Next to actual volume, there’s two stars. So, I expected to follow those asterisks, those two star symbols, to the bottom of the page and find out how they were defining actual volume and I never actually got to the bottom of that. They never actually define what they consider to be actual volume.
[00:13:30] While I directionally understand what they’re trying to say and I don’t think that they are lying, I also take issue with not defining what actual volume is. I do think that there is a bit of bias towards our base-10 numbering system. I think it’s interesting that there are 10 exchanges. It’s not nine exchanges. It’s not 11 exchanges that they say have actual [00:14:00] volume.
Finally, I think it’s a bit unfair to some really fantastic exchanges out there that are going out of their way to get method two compliance, for example, in Europe, or decentralized exchanges like Radar Relay … I guess those exchanges are trading in Bitcoin, but there’s a number of upstanding, fine exchanges and I don’t know if Bitwise would say that there’s enough volume in place [00:14:30] for it to count as actual volume. These are people with the best intentions that are doing everything that they can to be above board and I think, while those exchanges probably should not be used to feed this ETF, I don’t think it’s fair to say that there are 10 exchanges with actual volume. I think there’s more than this
I think the most unfortunate thing about this report, again, it’s not what they’re doing directionally. [00:15:00] As a Bitcoin holder, I really want there to be an ETF approved, especially one that holds the underlying asset, right? This is something that I very much want to happen for the industry, but I think what’s unfortunate is that this report came out and crypto Twitter did not question this. I am flabbergasted by the lack of scrutiny around this.
The number of people that are talking about this report that appear to [00:15:30] have not read it and the number of people that did read it that aren’t digging deeper into what this says from a space that tends to be incredibly, I think, critical and intends to be skeptical of a lot of things, I was surprised how much scrutiny this report did not get from crypto Twitter. I really expected people to crack this nut open a little bit and have a more nuanced [00:16:00] discussion, which is what I hope we have today is a more nuanced discussion. It stayed very, very surface level and so many people just walked away saying, “Oh, there’s only 10 exchanges that aren’t completely full of shit.”
Nathaniel: I think that there’s three issues that you just brought up in different ways that are worth maybe piecing apart. First, is a larger concern that’s not just about Bitwise, I don’t think at all, about methodological opacity, right? People not being clear about how things are calculated. [00:16:30] Basically, it doesn’t allow people to verify in an industry whose whole mantra is don’t trust, verify, right? This is the way that these things are calculated by the exchanges themselves, but it’s also the way that things are calculated in this report.
So, one issue that I heard was the opacity around the methodology for calculating volume. That’s what you want to see in that asterisk. A second issue is the concern that if people’s take away is these 10 exchanges and we don’t have a methodology [00:17:00] that can update things, we’re left with the implications stay stuck in a snapshot in time, right?
Nathaniel: We have this very, very, fast moment where we just say those 10 and then at what point do we update that understanding, right? So, that was the second piece is the snapshot in time becomes a heuristic for moving forward. Then, the third is this idea of the replacement of one trusted actor with another and I think it’s important to note that Bitwise is certainly asking the SEC to trust them, but they weren’t asking the crypto [00:17:30] community to trust them. They were making that appeal directly to the SEC and it’s a very different thing.
The third concern that you had and the response to this was this idea that we just go from one trusted actor to another. It certainly, again, this is not unique to Bitwise. You could easily see how a company could try to just be a better and more respected actor than an existing company with regards to information and transparency, but if [00:18:00] there isn’t that first detail, that transparency around methodology, you’re still in a trade trust of one for another.
Those are three things that I hear. I thought it might be interesting if you wanted to dig deeper on any part of them.
Clay: Yeah. No, so I think all three of those are spot on and that’s a really fantastic summary of my concerns. The first is above and beyond pointing out things that are obviously [00:18:30] incorrect about some of the exchanges that were highlighted as being bad actors like CoinBene, they did something brilliant, I think, from a marketing perspective, which is they showed screenshots of trading screens and annotated those screens in a way that made it almost self-evident that shenanigans were taking place on these exchanges. Then, they jumped from that to saying [00:19:00] that there’s only 10 exchanges with actual volume.
So, I would have liked to have seen some intermediary steps. Is there a methodology that can be generally applied? Are there some heuristics and ways of thinking about this data that allow us to continue asking the questions and continue scrutinizing these 10 exchanges? Maybe one of them [00:19:30] flips over and does become a bad actor at some point or there’s enough suspect trading activity on one of these 10 exchanges that are flagged as having problems, how do we know when problems start cropping up on those exchanges? How do we know?
It would have been great to come away with this report not just with examples of how exchanges are screwing up, but a [00:20:00] broader standard that can be generally applied. I think the second thing is what you mentioned is that if people do walk away with this saying there are 10 exchanges that are good actors, these 10 exchanges, I think that does risk letting our understanding of what’s happening be stuck in whatever happened to be true on the day that this report was published.
I think the final [00:20:30] way of summarizing this was that we’re really talking about moving from one trusted actor to another, so one of the big takeaways that people came away from this was, CoinMarketCap is really screwing up and they’re displaying inaccurate data. There’s a whole lot of problems in that, so let’s perhaps move to something else that only includes the great exchanges. I think there’s a problem with that.
The first problem [00:21:00] is that why was anyone blindly accepting what CoinMarketCap deemed to be true as true? This is probably a whole discussion topic in and of itself, but we regularly at Nomics have people write in and say, “You’re showing a different volume number for Binance than CoinMarketCap is. I found a bug,” or, “You’re quoting a different price for this asset.” [00:21:30] For example, Tether. We could spend a whole lot of time talking about Tether. “You’re quoting a different price for this asset than they are. I found a bug.”
The assumption there is that simply because CoinMarketCap is saying that something is true that it is true. In those situations, I like to refer people to our methodology docs and ask them to get similarly detailed methodology docs from CoinMarketCap. They, at this point in time, don’t provide them. They’ll give you a few sentences, but [00:22:00] I think a lot more goes into price.
One, don’t blindly trust these sources, but also to CoinMarketCap’s credit, what people love about places like CoinMarketCap is that they have more tokens and more exchanges than everything else. I think if they are going to be archiving what the space is doing, they can’t simultaneously include everyone’s coin and everyone’s crappy exchange [00:22:30] and vouch for the accuracy of all the data that’s coming in. At the end of the day, they’re reading data from these APIs. Those APIs are telling them something and there’s no way for them to independently verify that this is true other than independently auditing hundreds of exchanges, which is impossible.
I do think there’s a middle way and that’s what we’re pursuing at Nomics, but it basically comes down to methodology. Are there broader, open source methodologies for tracking big volume that are deeper [00:23:00] than taking pictures of screens that obviously have shenanigans on them? Can we apply that so that we can have an updated understanding on a daily or monthly basis about what’s happening? And, let’s not make the mistake of going from one centralized curator like CoinMarketCap to another that claims to have better data. Let’s see if there’s another approach [00:23:30] that can exist there and I think there can be.
Nathaniel: When I look at that report, again, I think the context is important. They were trying to educate regulators about what was really going on in the market and I actually think that if you take the first 40 or 50 slides of that as a 101 course in how to understand and spot these things, it’s phenomenally valuable, right?
Nathaniel: That second-order benefit of that report is that I guarantee you that there are a lot of folks who, if they dug into it, who didn’t really understand when people post charts [00:24:00] because that’s just not what they do that really started to get and understand what this thing was. Even by the end, they probably felt like they could spot weirdness.
So, what I wanted to do, and when we started talking what I started to get that was so valuable, was your understanding of the different categories of data that are available. When we’re talking about data from exchanges and what they provide, we’re not talking monolithically. In fact, the difference in the types of data that are available actually has a huge amount of [00:24:30] impact on the quality. So, what I’d love for everyone is for you to dive a little bit into what exchange data actually means and what separates really good exchanges from the less good actors.
Clay: This report, obviously, affected me. I think there’s no data provider in this space that deals with exchange data that wasn’t affected by what this report was saying and also the reaction to it [00:25:00] on crypto Twitter and other places. First off, I think the report on its own, and given Bitwise’s intent, I think the report was spot on. I don’t take issue with the report taken on its own and considered with respect to what it was trying to accomplish as part of an application for an ETF. It crushed it.
I think my issue is with the conclusions that [00:25:30] people have drawn after reading it and a lot of the knee jerk reactions that have followed. Let’s do a little bit of crypto market data 101 and explain the three types of pricing data that you can get from exchanges. Let’s actually ignore order-book data right now. We’ll just talk about data that directly influences [00:26:00] price.
There’s three types of exchange data. The first is raw trade data. This is data that describes each and every single trade that happens on every single trading pair that happens on an exchange. If you think about this data as a pyramid, let’s take Coinbase Pro, for example. Coinbase Pro has [00:26:30] an Ethereum to Bitcoin trading pair. Coinbase Pro was not quoting you Coinbase Pro’s version of what Bitcoin costs. There’s a whole bunch of trading pairs that include Bitcoin, but they’re not telling you what it generally costs. They’re telling you what trades were executed on different trading pairs in different exchanges.
You have data about specific trades that were actually made. Above that, you have candle-level data. [00:27:00] Let’s start off with trade-level data, which is individual trades executed by individual people. Above that, you have candles, which describe for a given period of time what the open-close high and low of a price was of what a crypto asset was against a specific trading pair. For example, if you’ve got Ethereum to BTC, you would be getting the open-close high and low of Ethereum [00:27:30] priced in BTC with this candle-level data.
Then, above that, you have what is known as ticker data. Ticker data is the crappiest pricing data available anywhere. Ticker data tells you for the time that a ticker was requested, it tells you what the price was 24 hours ago, what the price [00:28:00] is now, and what the volume is. It’s important to note that there are these different levels of granularity and most of what CoinMarketCap and most of what the majority of exchanges are reporting is ticker-level data. Meaning that when I stand back and look at all of the explicitly named bad actors in the [00:28:30] Bitwise report, 100% of them are only providing ticker-level data and they’re only providing it on a current basis. They’re not providing it historically.
What I found was if you take these two extremes you’ve got, on one extreme, you’re getting every single trade on every single trading pair going back to the beginning of that trading pair. So, [00:29:00] you’ve got super high granularity data that’s historical in nature. Then, you’ve got ticker-level data, which is not historical in nature, where they’re giving you volume for 24-hour periods. They’re not even giving you the open-close high and low. They’re just giving you opening price and closing price. Those are really the two extremes and when we looked at this report, what we saw was that the good actors were giving historical trade-level data and the bad actors were providing [00:29:30] not very granular data and not providing it historically.
One might think, why is this? What could be the cause of this? It’s probably on some level, it’s correlated with just an exchange operator’s ability to provide an exchange. That if you are thorough and good at what you do, then it’s very likely that your market data API for an exchange is going to reflect generally [00:30:00] how you operate. So, that could be part of it, but another part of it could be that if you are spoofing trade volume, you probably don’t want to provide highly granular data and you probably don’t want to provide it on a historical basis. Because that opens you up to scrutiny. It opens you up to being examined.
It’s not the case that if you do provide [00:30:30] raw trade data on a historical basis that you can’t fake individual trades. That isn’t the case. What I’m saying is that when I looked at this report, there seemed to be a correlation that we’ve noticed internally between the level of granularity and the willingness to provide historical data with good behaviors and the unwillingness to provide a history and the unwillingness to provide granularity with bad activities. [00:31:00] These things seem to be roughly correlated. They’re certainly correlated with transparency.
Nathaniel: Is it fair to summarize as saying that basically, it’s harder to fake things the more data that you provide? So, there’s a natural disinclination and correlation between that sort of bad behavior and just a lack of data.
Clay: Yeah, that’s a fantastic high-level summary and I wish I would have just said that right out of the gate. [00:31:30] It’s like the IRS. The IRS can come to you and you can fake all kinds of documents when you’re being audited. It’s just harder to pull that off. You just have to be a lot more thorough because eventually, if you do provide everything that they’re asking for, they’re going to catch inconsistencies and irregularities. It’s just a lot harder to do.
It reminds me of that. There have [00:32:00] been companies that have committed enormous amounts of fraud being audited yearly, public companies, but eventually, they seem to be caught because when you provide a lot of data and granularity, that tends to catch up to you. So, if you are a bad actor, that’s probably something you don’t want to be doing. You don’t want to be providing full history and full trade-level data.
Nathaniel: Yeah, I [00:32:30] think it makes sense. Almost by this logic then, the natural … Let’s go back to the questions or concerns. Again, I want to make clear that these concerns are the concerns with our community’s response and the spirit of this conversation is the right way to respond or at least arming ourselves with information to respond better. If we’re saying that we want exchanges to be better and to have more accurate data to make [00:33:00] better decisions about trades and everything else, part of what we should be demanding is understanding or asking exchanges for better data, right?
Clay: Absolutely, and that is part of our quest. It absolutely serves us because when we have better data, we can serve better data to our customers, particularly our institutional customers. But, it also is good for the space because [00:33:30] if you can make a high degree of detail and history available about what’s happening, it allows anyone to ask questions. Coming back a little bit more towards the side of the plight of Bitwise is because these APIs don’t provide a ton of granularity, you can actually get more detail about what’s happening by looking at those screens than you can from their [00:34:00] APIs.
The type of analysis that you would need to do in order to conclude that the volume is being spoofed is not analysis you can do by looking at tickers, really high-level data. In order to do that kind of analysis, you would need more data, which is available, ironically, in the trading interfaces of these exchanges but not exposed via an API, which it’s absurd to me. [00:34:30] But actually, it’s not because we all can guess what’s happening on these exchanges.
We hear that people want data from these specific exchanges where maybe volume is being spoofed, but our goal here is to give people filters that they can place on top of this data while not taking any of it away. Hopefully, over time, the approach is to have a variety of ways [00:35:00] that intelligent people can decide to exclude things that they don’t want to see. We’ll have a default version of this where we provide our recommended filters, but if someone wants to not use those filters or apply a different filter, right now, I think it’s going to be pretty rudimentary. But, in the long-term, perhaps we can use machine learning or other more sophisticated filters to consider everything but then exclude [00:35:30] what you don’t want.
Nathaniel: It’s also worth noting, again, going back to the context of the report that they’re talking about a specific asset in Bitcoin, right? And, they’re trying to get to the point of how they were going to calculate their methodology for their ETF specifically. This wasn’t just a frivolous exercise, there’s a very specific intention to make their approach feel like the right one or an acceptable one rather, in the context of the SEC.
But, when we’re talking about [00:36:00] exchange data in general, we’re not just talking about Bitcoin. We not just talking about the level of screening you needed for an ETF. We’re talking about a huge variety of decisions across an extraordinarily long tail of assets that trade all over the place. So, I think part of what you’re responding to, and I’ve seen other people respond to as well, is that even if you were fully onboard with this methodology as the right one for Bitcoin, there’s a huge number of other assets out there that are just different. One thing, I think, explicitly that they didn’t get into is decentralized exchanges, which are [00:36:30] growing, certainly, as a place where Ethereum and Ethereum pairs are trading.
So, I guess one other piece that’s interesting to explore a little bit too is just that we’re not talking just about Bitcoin if we’re talking about all of crypto and so that there may be a broader conversation about data transparency and how it will be necessary to solve the full thing.
Clay: I guess what has emerged from your last point about context is that perhaps the conclusions [00:37:00] that crypto Twitter and others have come to after reading the report are fair conclusions to come to if you only want to price Bitcoin. But, they’re not great conclusions to come to if you want to price Dai or if you want to price Tether or if you want to even price Ethereum or especially if you want to price an asset like 0x. The [00:37:30] same criteria just don’t hold up and would, frankly, be impossible.
If you were going to decide that you were going to perceive the entire universe of crypto assets only through these exchanges, that means that you would eliminate almost all volume for most crypto assets that aren’t in the top 30 and it means that for many of those top-30 assets by market cap, [00:38:00] you would likely only be using Binance’s data to price those assets and to determine volume, which I think is a mistake. I think if you are going to price an asset, you should use a few of the better actors and then price it on a volume-weighted basis.
I think once you start generalizing based on the conclusions of this report for the entire cryptosphere, I think you start coming to conclusions that [00:38:30] are probably not conclusions I would come to and that, in some cases, result in different methodologies being applied if you’re going to use one set of criteria to price assets that are in these top-10 exchanges and another set of criteria for pricing assets that aren’t in these 10 exchanges. I think now you’re using different methodologies to price different assets, and that probably isn’t good for an aggregator to do.
Nathaniel: Do you think there’s a moment [00:39:00] to try to shift the conversation and put the burden of community pressure on exchanges to provide different types of data? Is there a moment where if the conversation we’re having is about providing historic trade-level data, that you actually see certain exchanges start to behave better or is that kind of just a pipe dream?
Clay: Certainly, that’s what we hope to accomplish at Nomics. I like to [00:39:30] think about filters versus curation or searching versus curation. It’s like the difference between Yahoo back in the day and Google. Yahoo’s role, to a large extent, was to try and include the best resources and to rank them by hand in terms of importance. Google came along and said, “We’re going to apply this algorithm and, [00:40:00] in some cases, we’ll let you apply your own filters, but we’re going to include everything. If you make it very clear with your long-tail search request that you’re looking for something in particular, we will absolutely give it to you, but we’re going to have everything and we’re going to filter on top of this.”
Another thing that Google did that was really smart is they said, “All things being equal, we are going to rank better [00:40:30] sites from a data perspective above sites that aren’t performing as well.” One example of this is site speed. They’ve come out and said that page speed is a clear ranking signal for them. So, all things being equal, faster pages will outrank slower pages. Once they did that, anyone who cared about SEO was pouring [00:41:00] a lot of resources, a ton of resources, into page speed. Google, almost by themselves, made the entire web faster.
If you go to amazon.com and you use their Lighthouse page speed scores that Google provides, Amazon beats almost everyone because they care a lot about ranking in Google. They’ve done the same thing with meta tags, with title tags, with site maps for websites. [00:41:30] Google has done so many things that effectively say, “If you follow what we believe to be best practices, we’re going to let you bring that advantage when it comes to the search results.”
So, that’s our philosophy as well in our approach is to say, “Hey, can we do things that incentivize exchanges to provide better data. Because just demanding [00:42:00] it, in some kind of general sense, isn’t really helping exchanges. What I think it important to note is that unlike the New York Stock Exchange and the NASDAQ that have a monopoly on market data and can seek rent on top of that data, crypto exchanges are close to doubling every year. There’s a lot of them cropping up here and there, [00:42:30] and their market data, in a lot of ways, is a distribution channel for their product, which is trading pairs, ’cause they make money anytime someone trades across one of these trading pairs.
They’re trying to distribute it as far and as wide as they can and it really behooves them to get places like Nomics and OnChainFX and CoinMarketCap, et cetera to provide better data. I guess what I would call for is [00:43:00] for users and for the industry to tell exchanges that they want transparency, that they want history, that they want to see individual trade-level data. It’s not only going to help them with websites like ours, it’s going to help them attract market makers and it’s going to attract a whole lot of other actors that are going to find more utility in the exchange because that data exists.
Nathaniel: There is actually a pretty interesting and kind of obvious analogy here [00:43:30] between what got people so excited about Bitcoin in the first place, is the ability to go back in time and see a timestamp of everything that had ever happened. It was this thing that was fundamentally different in terms of opacity, in terms of transparency, than what existed. That just instantly differentiated it and it made … In some ways, you could argue that the entire point of the security system is to [00:44:00] preserve that, to not allow people to attack that. So, to some extent, it’s ironic then, I guess, a little bit that the most important set of businesses and institutions built on top of what Bitcoin did and came after don’t have that same level of transparency when it’s available.
It’s not a surprise to me, I think, in some ways after having dug in and learning from you about the different kind of categories of data, that the best actors in this space are the [00:44:30] ones who provide that level of transparency. It does seem to me there is an opportunity to hold the exchanges that we use to the same standard of the underlying assets that got us all here in the first place.
Clay: Yeah, yeah. I completely agree. People would really, really, really be upset if we said, “All right, all of a sudden, going forward, Bitcoin is just going to tell you what addresses have what amount but we’re not going to show any history and you’re not going to be able to reconstruct historically what [00:45:00] happened with the ledger to get it to its current place.” That would really piss people off, but we’re accepting this with exchange data every day from almost all the big exchanges.
Nathaniel: Do you think it’s just a part of the natural evolution from a less mature to a more mature industry?
Clay: I think that’s part of it. I also think that most people don’t interact with exchange data very much at all and, [00:45:30] for the most part, they just care about whether or not they can log in, make the trade that they need to make, and then pull their assets off that exchange. They care a lot more about the technology that underlines their bags than the technology that they use to make one trade prior to pulling everything off.
At the end of the day, people care about do they have the token I want and will they accept the token I have for it and am [00:46:00] I okay with the price and are the order books deep enough that if I execute this trade, it’s not going to move the market significantly in a way that disadvantages me?
Nathaniel: Why would an exchange try to shove more volume? It comes down to their business models, right? What they can show or what they can do in terms of listing fees or what they can get in terms of advertising. Those things, it’s a pretty dense network of how [00:46:30] that set of incentives influences other things. That set of incentives influences people to create more coins, which influences people to spend more money on advertising. All of these things have impact on the market, so I think that if you take an individual logistic view of does it affect me, that exchanges are engaged in this [inaudible [00:46:51] or do I care?
It’s easy and probably honestly reasonable to not spend too many mental cycles on it. However, if you’re looking systemically from what [00:47:00] the state of our industry is that if you have any kind of sense of wanting it to mature and be better and be focused on the things that are real and that are good, there’s a lot of good reasons to care. I think that’s the interesting moment to me about this conversation, why I was excited to do this with you, is that sure, this is not going to impact the day-to-day, but you know what? It is because if Bitwise’s ETF proposal goes through, there’s going to be implications on the price of your Bitcoin.
I think, from my standpoint, if I could leave people [00:47:30] with anything, and, hopefully, this is a chance to dig in deeper, it’s that these things actually do matter fundamentally in terms of the state of the industry in ways that are perhaps not totally obvious but are, in fact, incredibly important. Not addressing these kinds of issues is a sure fire way to have us be in a constant one step forward, two steps back kind of modality.
I was really excited to dig in with you on a deeper level about these things. I feel [00:48:00] like I have a lot more to learn still, but I think this is a good moment to do that sort of learning.
Clay: I want to correct what I said. During a one-hour time span, does Joe Bagholder really have an incentive to care about these things? Probably, not over that time horizon, but when you consider that it’s these types of shenanigans that have resulted in an ETF not being approved thus far, that’s really a problem. [00:48:30] When institutions feel like they can’t get in or trade in a crypto asset that might be really interesting but simply isn’t available on a mainstream exchange that’s doing things right because all the others don’t have an audit trail that will enable an institution to get involved, that’s a problem.
When we look at this space, there’s about 12 big exchanges [00:49:00] that have historical raw-trade data that they’re making available. Of all the ones that we look at that have any kind of significant volume, and I realize that’s completely qualitative and subjective, there’s probably another 24 exchanges that have candle-level data down to the minute that they make available in real-time and they typically don’t make historical [00:49:30] candle data available. Then, all the other exchanges … We went from about 12 with raw-trade data, another 12 with candle data, and another, I don’t know, 200 plus exchanges that only provide ticker-level data with no history.
I think a lot of times exchanges come up and they go to create an API and [00:50:00] it’s an afterthought. They think about their level of transparency and so they just look at other actors in the space and they say to themselves, “This is good enough.” It’s because we aren’t holding them to higher standards, but I think the impact on the community is being felt right now with our ETF not being approved. Whatever crypto asset you care about, I can guarantee you that we’re likely to go through the same thing with an ETF for that because [00:50:30] there aren’t enough exchanges that provide auditable histories about what’s actually moving the volume for that coin.
Again, in a short time horizon, the incentives tend to be that exchanges will fake volume, but they won’t typically do it in a way that manipulates price because if they are too much above market or below market, [00:51:00] they’re going to turn away potential customers. But, they do want the listing fees. They do want to rank higher on CoinMarketCap. They do want to do a whole bunch of other things that probably gets them attention and it’s hurting the space. It prevents institutional money from coming in. It prevents all kinds of things that we need for the space to mature and for other people to want your bags at prices much higher than you bought them for.
Nathaniel: [00:51:30] Let’s try to just wrap up with a few high-level conclusions ’cause I think this is super cool. Again, I think this is meant to be another jumping off point in the same spirit as the original report was. One is, I guess, a take away for me is just around the actual data itself, that what we should be looking for from exchanges is more granularity of the data that they provide and complementing that, more transparency [00:52:00] around the methodology with which anyone who’s interacted with that data is interacting with that data, how prices are calculated, how volume is calculated.
That’s not about any one company or another, it’s about everyone, right? That’s one thing to take away. A second is to recognize that this is a big issue for the community and that it has implications that are specific and are felt now and that will hold the whole industry back [00:52:30] from maturing if not addressed. Then, I guess a third is just what better time than now to start engaging with this. There’s a narrative moment that Bitwise has kicked up, which I think is awesome, to have these conversations, to make it front and center rather than just going on and dealing with it.
I think it’s especially important as we get excited again with price action, which we’re recording this right after Bitcoin had its big pop, which was [00:53:00] thrilling for people. But, it’s very easy for conversations about things like the quality of exchange data and the implications of wash trading to get swept under the rug as fast as they get exposed when all we have to care about is 15%, 20% jumps in a day. I think it’s especially important now without knowing what comes next, let’s use this moment while we have it to actually shine a light on this [00:53:30] area of the industry and make it better.
Clay: In terms of these high-level takeaways, there’s so much nuance about data. I would say that the takeaways are that there’s really two dimensions where differentiation can happen. The first dimension is on granularity and the second dimension is on history. You can have a high level of granularity and no history and you can have a lot of history and low granularity and then everything in between.
All the other points I think [00:54:00] that you made are right, that we’re happy with price action, but let’s not stop fighting the good fight when it comes to the standards that we hold, not only crypto projects too but also the entire ecosystem that makes the system workable. Yeah, I think that’s really important and I’d be remiss if I didn’t comment that what [00:54:30] we’re going to do around this, and it’s not the solution to everyone’s problems, but what we’re going to be doing is ranking our exchanges first by their level of transparency and second by volume.
So, a high transparency exchange will outrank another exchange with higher volume but less transparency. So, the first-order sort is by transparency. [00:55:00] Within that tier, it will be sorted by volume. For example, Binance is, I think, a great actor in terms of what they make available via their API and the kind of auditing that you can do of activity that’s happened there. CoinBene, CoinBene … I don’t really know how people pronounce it … but the quintessential bad actor in this Bitwise report, on some days, reports more [00:55:30] volume than Binance. So, CoinBene would never be above Binance because of the kind of data that they provide.
That’s how we’re solving it. I think it’s the first iteration of this. Thanks to Bitwise for actually surfacing this. I’m so glad we’re having this conversation as a product person and somebody who thinks about this stuff. It’s a great opportunity to differentiate. I think it’s actually a really great thing that they kicked [00:56:00] off this conversation.
Nathaniel: Yeah, it’s not every day that a meeting with the SEC can turn into an industry-wide conversation that’s directionally very positive and really important as well.
Clay: That’s it for this week. To sign up for our free crypto investing newsletter, listen to other episodes, or get the show notes from this episode, please visit flippening.com. I also invite you to check out the startup that funds this podcast, Nomics, spelled N-O-M-I-C-S at nomics.com. Finally, if you got value from this show, the biggest [00:56:30] thing you can do to help us out is to leave a five-star review with some comments and feedback on iTunes, Stitcher, or wherever you listen to podcasts.
Thanks for listening and see you next week.