Forkbeard - Indoor Positioning with Ultra-Sound

Transcript

• Narration 00:07
  

  The Mr. Beacon Podcast is sponsored by Wiliot, scaling IoT with battery free Bluetooth.

  Steve Statler 00:16

  Welcome to the Mr. Beacon Podcast Hope everyone is doing well. Gosh, it's been months we've been going through this COVID nightmare. But you know, one of the wonderful things is that we have the technology that we're using today to talk to interesting people around the world. I don't think we've ever spoken to a guest from Norway. And I know we've never delved into ultrasound, which is a major mission and one that I'm looking forward to rectifying. And we're, we're doing it with Wilfred Booij, who is the CTO of Forkbeard. Wilfred, welcome to the show.

  Wilfred Booij 00:57

  Thank you, Steve.

  Steve Statler 00:59

  Um, so you were part of another company that was dealing with ultrasound before for better how old is for bid? And what's the relationship between ForkBeard and this other company.

  Wilfred Booij 01:13

  So Forkbeard, there's actually quite a new company, we started it, I think, in January 2019. So it's just over one year and eight months old. But the sort of like that, we started actually, with 15 people. And those came out of another company that was doing ultrasound based indoor positioning, because it's called Sonitor. And we actually, we still send sharing offers with them a little bit, it's two separate companies now. And Sonitor basically, has been developing indoor positioning solutions for us primarily in healthcare. So it's basically an ultrasonic technology where you have transmitters in the infrastructure, and it's tag based. So basically, we have tags that receive ultra sound, and can then be used to do positioning of people and equipment. And the key strength is this high resolution, zonal capability. So the fact that you can actually tell whether a patient is in a bad and whether equipment can be associated with the patient and so on.

  Steve Statler 02:14

  And this is sonnet or the you're talking about Yeah, this is sort of Yeah. Okay, so you have the tags, the receivers, the the, the kind of the infrastructure is broadcasting ultrasound,

  Wilfred Booij 02:25

  correct? Yeah. Yeah. And, and one of the things that we always dreamt about in that company is to do to do 3d proper 3d positioning. So rather than zonal based system, which is what Sonitor has today, and which is very suitable for for in the healthcare business. The aim was to have a system that basically, you know, where we can do software zoning, so sort of fundamentally 3d accurate 3d positioning. And then, in software, you can define your zones instead of your rules and so on. whereby you can actually decouple the big thing there is that you that allows you to decouple your infrastructure from your use case, which of course, if you want to scale something is really important to the fact that you can basically set up an infrastructure and say, it's like Wi Fi, now you have common communication, now you have positioning throughout your building, and then you can basically use it forever you would like to use it for so it needs to be accurate, low latency, and, and persuasive in the in the in the whole, whole building. And so, so that's basically what we set out to do in Forkbeard. And one of the main other realizations was that let's let's do it using smartphones. So let's go the tech route, would instead use a user's smartphone, and basically app enable the positioning. Okay,

  Steve Statler 03:51

  so one of the differences is Forkbeard, has this smartphone components. And in this case, the devices that you so do you just sell one kind of device or transmitter. Yeah,

  Wilfred Booij 04:04

  at the moment, we've we've got one, one device, which we call the ultra beacon for ultra-sound beacon. And it's it's really a combined Bluetooth beacon and ultra-sound beacon,

  Steve Statler 04:16

  how large are these devices,

  Wilfred Booij 04:18

  they're actually quite small, they're about the size of a Apple TV box. And the size is really dictated by the the the four AA batteries that were used to power it. Okay. And it's quite interesting because we basically based on the experience we have in Sonitor, we had sort of like a target for battery life and we basically said needs to be more than at least three years, but preferably 5-7 to seven years. So it's sort of like a you know, set and forget the help of new releases from you know, from chip manufacturers were able to enter two to 10 years of battery life on for AA and that standard. That's not true. Just sort of like a particular configuration, that's any configuration. So that was a really major achievement.

  Steve Statler 05:06

  Yeah. 10 years is is good running off batteries. So people don't need to worry about wiring and, and so forth. So, and typically, how many of these do you need to achieve a given level of location fidelity?

  Wilfred Booij 05:24

  Yeah, since it says it's a solution that basically wants to achieve 3D. And the minimum quantity that you have to be within reach of is three at the moment. So, so that sense of sort of like a minimum density of devices. That depends a lot on how, you know how fragmented the the environment is. And but we basically there's a sort of like a rule of thumb, we're saying around one for every 30 square meters. Okay, design guide line.

  Steve Statler 05:57

  That seems pretty good.

  Wilfred Booij 05:59

  It's pretty good.

  Steve Statler 06:00

  Yeah. And if I have 130 square meters, then what kind of location accuracy am I getting?

  Wilfred Booij 06:09

  It's about about a feet of accuracy in, in the two XY dimensions, and then it's a little worse in Z dimension, it's about probably two feet inside dimension.

  Steve Statler 06:19

  Okay. All right. I guess so when does that matter? I mean, if you're tracking a bone, wouldn't seem like that matters too much that?

  Wilfred Booij 06:28

  No, no, I mean, it's sort of like if you get into how atriums and that sort of stuff, it's interesting to have a good set dimension and then to fetus is more than adequate

  Steve Statler 06:39

  to be more so positioning this against other technologies just bear Bluetooth, you're much better. You're probably slightly better than angle of arrival. I maybe the angle of arrival for Bluetooth guys would would argue with that. But it seems like you're a bit better, and you're not quite as good as ultra wideband. Is that a fair characterization of where you sit in the Pantheon? Yeah, I think so. Like, yeah.

  Wilfred Booij 07:10

  Okay. And then, of course, we have the advantage that it's a fully battery powered infrastructure. So if you look at the cost of installation, then it's quite dramatically different. And in in itself, it's a very cheap device, you know, it's not like an expensive unit to manufacture at all. It's essentially a Bluetooth beacon with with ultra-sound capabilities bolted on.

  Steve Statler 07:30

  So for budgetary purposes, what what should people budget for if the if they if they want to buy 100 of these units, what should they budget for as a per unit price.

  Wilfred Booij 07:43

  So we typically we try to price by square meter. Okay, so it's sort of like targeting as sort of like a couple of dollars per square meter as our as our as our pricing for installation, which is really quite quite good. Because you know, if you look at a supermarket and average sized supermarket is about thousand square meters. So So in principle, you should be able to equate that with our solution for a couple of thousand dollars.

  Steve Statler 08:12

  couple of thousand dollars for roughly,

  Wilfred Booij 08:16

  it's about probably about, so that would be 1000 square meters, if you divided by 30, there will be 30 to 40 units.

  Steve Statler 08:24

  Okay. That is pretty low cost. And, you know, you know, the bane of Bluetooth, existence is metal and that sort of thing. Presumably, this is you're less concerned about that, although, yeah, how much? I mean, we think about line of sight, what what role does line of sound have your?

  Wilfred Booij 08:48

  Yeah, it's? That's a very good question. And it's one of the reasons why we actually went for notary signing in the first place. And because of course, if you look at a pressure wave interacting with the environment, it's actually extremely predictable compared to an RF wave. So you have an RF wave, especially 2.4 gigahertz, if it impacts on a particular piece of material, you've got all sorts of mechanisms gone, you've got basically back scattering, you got absorption, you got back scattering a couple of 10 centimeters back. So so the one of the I think one of the fundamental problems with RF is that it's very difficult to to to basically get an idea of what your line of sight is, because you, even if you could do path decomposition, it would be extremely difficult to say, Well, this is the line of sight. And that's one

  Steve Statler 09:36

  of the biggest briefly used back scattering, not everyone will be familiar with. Yeah, so we've basically impacting on a service, you can basically have a couple of fundamental interaction and scattering is just it's just use the face information. And the wave is coming back at you with an unknown face in a way. Right. And, and one of the really nice things with ultrasound is that You have none of that. So when basically wave hits and ultrasonic wave or pressure wave hits a surface, it's most of the time perfect reflection. And what happens a perfect reflection is that all the information is, is basically intact. And so you do have another path, do you if you're

  Wilfred Booij 10:25

  Yeah, yeah. So yeah, yeah. So but what we found out, and we've done a lot of patenting on this, and if people are interested, they should should look it up. But it's, it's a method of, of basically path decomposition. Where you can basically, you know, when I, I have a transmitter in typical building, and it's, it's basically sending a signal out, and I'm within a couple of meters, we typically find six, seven paths, we can decompose all the paths, and use in fact, all the paths in the position.

  Steve Statler 11:00

  Oh, so So actually, these multiple paths through this part of the composition, they help rather than hinder?

  Wilfred Booij 11:06

  Yeah, yeah. Yeah. So So future versions, we could probably, you know, you could be looking at doing positioning only using a single single transmitter in the room. Okay, by using that buffer decomposition.

  Steve Statler 11:20

  So it sounds like the primary use case that you're serving here with forkbeard. Is IPS, indoor positioning systems, although presumably, this works outdoors as well.

  Wilfred Booij 11:33

  Yeah, no, it is it is firmly focused on indoor positioning. So the whole, the whole mission of the company is to basically take the GPS revolution indoors, we don't want to take GPS indoors, but we want to basically enable the world to have to put them to fill in the blanks in a way indoors. And of course, it needs to be better than GPS in many ways indoors, because your dimensions are shrunk. And you're interested in, in more resolution than you might be outdoors. So that's the aim of the company.

  Steve Statler 12:07

  Very good. And this presumably then relies on you having an app? I mean, what's the point of doing it if you don't have an app? So you have a Forkbeard aware app that has some API's that are presumably listening to the microphone? How do you deal with getting the apps in the prison? because presumably, to do this, the app needs to be in the foreground, you can't have? Or can you? Can you have an app that's listening to sound in the background? Or do they have

  Wilfred Booij 12:37

  got some good news there, the app is on the App Store and on the Play Store, so people can download it, it the challenge now is to find a site at a venue that supports it, you have to travel to Norway to to find that

  Steve Statler 12:51

  comes to Norway, Okay.

  Wilfred Booij 12:53

  So we've got to install a couple supermarkets and offices. But the Yeah, the really interesting thing is that this also works in the background. And you can actually have the phone in your pocket and get the same quality of off of positioning as you would have when you have the app in the background.

  Steve Statler 13:11

  Amazing. Um, but so what happens if I have like rebooted my phone? It can still, it can still work in the background?

  Wilfred Booij 13:21

  No, no, I mean, it does run as normal app. But we basically, because we don't, because basically sound is enabled is enabled in the background, because we avoid applications. It allows you to basically use you sound in the background in both iOS and Android

  Steve Statler 13:42

  and other permissions that I have to opt in to to allow us. Yeah, so you're not you're not running afoul of the privacy thing someone has to say yes, it's okay for this. So I thought I saw something about Bluetooth in what you did as well. Do you combine Bluetooth in any way? Or was I just miss reading it?

  Wilfred Booij 14:03

  Yeah, absolutely. Yeah. So, the, the the concept is, so so one of the one of the great things about ultrasound is that you can do very reliable path decomposition, and you can do very reliable distance measuring. And, but the bandwidth of it isn't great. So, you have very limited bandwidth on the phone is really limited, I mean, basically, our signal spectrum is you know, it has to be all about 20 kilohertz and then the microphone start to drop around 22 kilohertz, so, you effectively only have a kilohertz or so to play with. So, the amount of information can send from a single beacons is very limited. So, so, what we do is we actually use a combination of Bluetooth and ultrasound to to give enough uniqueness to the to the beacon. So, and in addition, the, the the, you also can basically use some some of the lightning and thunder principles. So the fact that you know, an RF If you're traveling much faster than and then a pressure wave allows you to do distance measurements.

  Steve Statler 15:05

  Okay, so in this case, the Bluetooth is the lightning, and we see the lightning before we hear the thunder. And then obviously your ultra-sound is the thunder, although we don't hear it, can you? Can dogs hear what you're transmitting?

  Wilfred Booij 15:20

  Yeah, it's even even even some some kids that are very low HRV are actually capable of hearing above 20 kilohertz, but it I think it drops off within the first 18 months or so. But given our experience in the field, what we did is that not only is it sort of like, encoded in a very special way sort of doesn't create lots of harmonics, but it's also really low level. So most of the time, you can't actually it actually in the noise floor, it's a little bit like ultra wideband in that the signal level is extremely low.

  Steve Statler 15:55

  Okay. So dogs won't stop barking, babies will start crying, there's nothing.

  Wilfred Booij 16:01

  No, no. So the idea you're looking at SPL levels of 60 db, or less so, and 60 db, that's, I think, if you drive any car, you're already up in this sort of 70 7080 DBS 60 days. Yeah, he's sort of like office now.

  Steve Statler 16:19

  So what about kind of the time dimension to this? Am I am I getting? Can you kind of quantify the the accuracy over time? If I'm like sprinting through the shopping mall? Will you? How many times will you see me versus if an old person is? is making their way? Slowly?

  
  

  Wilfred Booij 16:46

  Yeah, I think it's a really good question. And and the, the good news is there that we typically have between five and 10 updates per second. And the other nice thing is that there is something called the Doppler effect that most people might be familiar with, you know, when you're passing out the siren, you get this shift in frequency. Yeah. And what that allows you to do is, it allows you to measure velocity at the same time. Ah, not only do we have distance, but we also have velocity of, of the phone. And not only do we have the velocity along the path, we have the philosophy and all the paths.

  Steve Statler 17:23

  So this is fascinating. So maybe if I have some kind of point of sale promotion, you can see people not only passing it, but people slowing down to look at it, you can get the impact of engagement, because, okay, I put the sign by the front door, yeah, everyone's gonna walk past sign, does that mean that sign is super successful? Or does it just mean that people want to get through the front door? But if you can say, Oh, you know, we had some kind, it's sort of the almost like dwell time on the kind of something a little more subtle that you could potentially deliver is that?

  Wilfred Booij 18:02

  Yes, yeah. So you can basically have a sense of direction instantaneously, which is also really interesting in in position, of course, so so you can basically have a, you know, pass through door, and you can basically tell what the direction of the pasture is without having to have multiple position events. Okay. Which is also really interesting. Plus, of course, if you're into sort of like Kalman Filtering, which we are, and then having the state and the derivative of the state, so having the position and the velocity really helps in getting and getting higher accuracy.

  Steve Statler 18:35

  So he said something about filtering, and I miss

  Wilfred Booij 18:38

  Kalman Filtering. So this idea that you have a state estimate, where you have multiple observations that are related in some way that you can easily model. And of course, here, it's very clear, you have a basically a range, and you have the derivative of the range. So okay, so having both velocity and and distance to an in between the smartphone, and the transmitter allows you to improve the accuracy dramatically.

  Steve Statler 19:04

  So how can I consume all this information? And it seems like there's potentially a lot there. How do I, how do I get it? What sort of app so you see that I believe,

  Wilfred Booij 19:17

  so the the principle is that we're not an app company, we're basically delivering the tools for others to build apps. So what we have is an SDK, we have multiple SDKs. And those SDK is basically do all the heavy lifting. So what you actually get is a is a an API, which is very similar to GPS. So you have a start, stop, and you have a position push. So you can basically subscribe to a position updates in the app. And then basically, every time we get additional data, and you can also define some parameters and say, I wanted a high resolution update or a very low resolution update. So so it's very much like basically what we did is we looked at the API is that We're available within the operating systems and said, This is what from developers? No, this is what we want to deliver to the developers. So our experience is that, you know, people can basically get an app going within a day or so based on the SDK.

  Steve Statler 20:16

  And how, how's that guy in terms of app integration? Do you have demo app? So there are other apps partner apps that you've integrated? Yeah.

  Wilfred Booij 20:26

  Yeah. So like I said, that we have a four bit lira app on the App Store and Android Play Store. And that basically just shows us a demo app, it just shows the capability of the elder solution. And then there are a couple of custom apps which unfortunately, can't disclose at the moment, because they're still working in development phase. But some of them are office type type environments. And we have two supermarkets that have the installation in Norway. And they're the the use case are very advanced in that it's basically product. So it has a complete product database. And it has multi points, navigation to do your shopping in the supermarket. So you can basically, you can you can compose your favorite shopping list. And then it will actually show you the route to the supermarket. download any recipe and written and one of the big problems, of course, people have any recipes is that there are new ingredients that they never find. So this is the idea behind the app is that you basically it's a very good way of getting people to to be a little bit more adventurous in cooking.

  Steve Statler 21:34

  Okay, very cool. And where do you so it sounds like you're basically given GPS coordinates. So you're partnering with all of the mapping, you work with third party partners that can give you a map. And

  Wilfred Booij 21:49

  yeah, I can, I can actually live directly on top of, of the Apple and Google Maps, solutions. So So basically, what we also offer is an additional API and SDK that allows you to, to live on top of Google and Apple map with an additional layer. And then there is there's example code that allows you to basically easily draw paths and that sort of stuff. So we also actually have routing routing SDK,

  Steve Statler 22:18

  I was gonna ask you about that. So you can give people the fastest way to get to the cinnamon. The from the fruit pile? Yeah. supermarket. Yeah.

  Wilfred Booij 22:30

  And it's actually configuration free routing, so you don't actually have to draw any paths in advance. So he actually uses the the map of the store to automatically do the routing without having to specify paths. Before I write.

  Steve Statler 22:47

  The question I've got to ask, is about the name, can you you're probably sick of telling this story, but

  Wilfred Booij 22:54

  no,

  Steve Statler 22:56

  he's got a very nice logo. I really like the logo. It's sort of resonates from Bluetooth in that you've got letters in this kind of round thing that looks like a map. Pin, but where does it come from?

  Wilfred Booij 23:13

  So the name is interesting we basically Of course, we were starting the company we said well, you know this there's Bluetooth and people associate Bluetooth with with positioning these days. Although really it's a communication technology so so we decided to go back in history and say and Bluetooth Of course in in Scandinavia is very well known role. Tom is a Viking King, one of the best known ones, and what

  Steve Statler 23:40

  was what was exceptional about if we're gonna get to the whole thing, then what was exceptional about Bluetooth as a kid that he got given this association with a communication,

  Wilfred Booij 23:51

  He was a smooth talker. He was very good at communication. And now the best part of the story is, and we as we started to think about Okay, historically who was the person that really Trump Trump to Bluetooth and it turned out to be a son up name was his song is Forkbeard. Okay. So unfortunately, not he was a good navigator. He was the guy that basically invaded England, unfortunately. I don't want to talk more about that. I don't think you could have named a company if it was possible 50 years after its birth, I don't think it would have gone down that well. You know, things fade and it's an acceptable name now.

  Steve Statler 24:42

  Yeah. Well, yeah. And appropriate given that you have some Bluetooth lineage in your in your in the makeup of your solution as well. Very good. Well, this has been fascinating. Any anything else that I've, I should have asked you about that. I think

  Wilfred Booij 24:59

  You did a tremendous job, Stephen in asking the right questions.

  Steve Statler 25:04

  Well, well, we'll end up on a high note then. So wilfer, boy, CTO of Forkbeard. Thanks very much for being on the on the Mr. beacon show. Thank you, Steve for giving me the opportunity. How should I pronounce your last name Wilfred?

  Wilfred Booij 25:26

  It's very simple. It's actually Booij. Booij, yeah. So okay, so it's free Shin, which is very close to English. Okay, it actually means boy.

  Steve Statler 25:40

  I was glad I asked you normally I haven't go and massacre people's last name. So that would have been pronouncing the J. So

  Wilfred Booij 25:49

  for that in Dutch, they didn't have the the Y on the keyboard. So it became ij.

  Steve Statler 25:55

  Oh, so that's your ancestry is is frisian. It says northern northern parts of the Netherlands. So how did your family end up in? North? Oh,

  Wilfred Booij 26:07

  I moved to I so I studied in, in, in first the Netherlands to the masses in the Netherlands when I moved to Cambridge. And my that's where I met my wife was the region.

  Steve Statler 26:18

  I saw that you did your PhD there, didn't you? Yeah,

  Wilfred Booij 26:21

  yeah. Yeah. So also like, very nice. Yeah, it was very interesting. It was a fun thing

  Steve Statler 26:28

  on the cam, or

  Wilfred Booij 26:30

  something on the cam and really international environment. I was a fellow of golden keys college. So I actually got a fellowship there and dine with Stephen Hawking. Next, what was Stephen Hawking effect? So yeah, lots of lots of really interesting experiences.

  Steve Statler 26:49

  I went to a Polytechnic to do my computer science degree. And basically, as a result of a misspent youth, the time it wasn't a good place to study. But by accident, it turned out to be a great course. I was just every, you know, all the fundamentals. But that was really by accident. The time I had a girlfriend whose father was doing a, I can't remember what it was. It may have been a PhD at Cambridge. And I remember going there and having to wear a tuxedo to have dinner. Yeah. Hi, table service.

  Wilfred Booij 27:31

  Yeah, so as a fellow he basically have free dining every evening. And it's still you know, they serve the best wines and food is phenomenal. And then you see the poor students sitting, you know, just two meters away, and they get served a lot, not the same quantity of food. And of course, you see, are you I was used to sitting there. So it's, it was a very weird experience.

  Steve Statler 27:54

  And was it different? I guess your experience was different. You were slightly older than the

  Wilfred Booij 27:59

  Yeah. So I started as a grad student. So I did a three year PhD. And on the end of the PhD, I got this research fellowship. So it stayed on for another two years.

  Steve Statler 28:12

  Yes, yes. Did you ever sort of wish that you are the 1819 year old having that experience? So is it better? No, no. What a good one was?

  Wilfred Booij 28:26

  Yeah, I think it's very difficult to be Dutch student life. So that's, it's fantastic. Because you have an enormous amount of freedom. And at the same time, you have this, you know, rich tradition of quickie student behavior in Holland. So, so there's a lot more freedom than you would ever had on sort of like a UK University. So. So I actually took six years to take my masters and that was time well spent.

  Steve Statler 28:59

  Excellent. What a good one. As you probably well, you may or may not know, I don't know if you've ever watched any of the past episodes, but we have this tradition of asking people about their music tastes. Ah. So and basically the construct is, what three songs would you take on a trip to Mars sounds like this is is new to you. So I'll give you about it. But his first question, I guess, is is music important to his music part of your life when you were a student? Yeah, absolutely not. Yeah, yeah. Although I a lot

  Wilfred Booij 29:36

  less now than it used to be. But yeah, I mean, sort of, I guess, quite quite a set of music days now, which I'm to like, infecting my kids with.

  Steve Statler 29:47

  How would you characterize it? Oh,

  Wilfred Booij 29:50

  it's probably quite it's, it's, it's fairly mainstream. I would say it's, you know, it's the usual stuff, having been born in the 70s. So it's, you know, Hotel California, Steely Dan. Later advocating Lou. Yeah. quite straightforward. Nice easy listening. Yeah. And what jazz jazz has said is also quite. Yeah, quite interesting.

  Steve Statler 30:15

  So very similar to my own. So what if you had to pick three songs? What do you think?

  Wilfred Booij 30:22

  I guess, the Night Flight? Hotel California would say it's probably somewhere up there. Yeah. three songs actually would be more difficult. I guess it could be Killing me softly.

  Steve Statler 30:37

  Okay. So good. Sounds wonderful. Thanks very much.