[0:10]Hello, world. Welcome to the 203rd video on my channel where I'm building my own digital assistant named Shane, like Jarvis from the Iron Man movies and comics. Please consider subscribing to my channel, liking this video, and leaving a comment on if you play an MMO RPG. So I'm a big fan of massive multiplayer online role playing games, uh, or MMO RPGs, especially ones that have a real functioning economy in them. And so I started off playing Eve Online a long time ago, and I played it for about two years and it kind of became an obsession, and so I just quit Cold Turkey, but at the peak, I we were in a medium-sized what they call corporation with about 200 players. And the way EVE Online and now this game, the game I play now called Albion Online play, is the game releases a basic set of kind of information or code. And then the players gather materials, mind them whatever it's called in the game, and then create the items and materials that you need to play the game versus other games which just created it for you and there's almost an infinite supply. And so since there's not an infinite supply, there's a real functioning market where you can buy and sell things and it's very similar to kind of a hyper capitalist economy, right? And so I played EVE Online, we were known as Care Bears, uh because there are different levels of security in the games that allow you to play without fighting other players known as player versus player or PVP. And that's where I I'm not a big fan of fighting people and so real people, and so we just had this high security place and my job was to download all the market data because EVE Online let you download it directly from the game. And then all day long, I would just play in Excel, which is why Eve Online's called a Excel simulator. And so I've gone beyond Eve and now I play Albion, I play by myself, I'm not interested in a massive corporation because it was a a part-time job basically. And so now I just play casually, but to play the game efficiently, you need to know and understand the market. Now, the game does not have its own API, but there's an Albion online data project which tries to crowd source information to give you that API. And that's what we're going to do in this video. I have no intention of making a real application or competing with any other website, so this we're just going to explore the different things you can do. And basically we're just going to use requests to scrape the data and then use uh URLs, custom URLs to figure out the data we need. So we're going to start off by importing requests. Um we'll we'll skip this for now cuz we'll use this for historical prices, but the first thing we're going to do is look at current prices. So if you go to the Albion-online-data.com, you can kind of see the different options you have. So here is a sample URL that you would pull to look at the current prices either in a table view or JSON view of a T4 bag, a T5 bag, locations in Caron and Bridge Watch and qualities too, so I what's known as a 4 or 5.2 bag. You click on there and you get a JSON file of the prices. Okay? So that's an example and we can use that in Python. So the first thing we're going to do is build the URL, right? So the first thing is the price URL equals this, right? Basically, the first part of this, um, URL right here up to prices. And then I've created a list and these are the formatted, properly formatted titles of the two resources that I focus on. So let's let's check out what that is. So, uh, I have a URL right here that goes into the data dump, so let's look at the items text. All right, so we can see that let's let's look for wheat soup. Wheat soup, there we go, and generic or, you know, basic tier zero, T3 meal soup is what they call meal soup. And we have that here, T3 meal soup, make sure you have the underscores. And then I also am a wood gatherer, so let's look at cedar planks. And generic cedar planks are just called T5 planks, make sure it's capitalized, basically copy and paste it from, uh, here. All right, and then location. So I want to look at Lihurst, that's where I hang out at. I know it's not optimal, I'm not really interested in the highest percentage, the most efficient, right? That's how I kind of lost my mind in Eve Online is scraping out percentages of, uh, percentages, you know, fractions of a percentage just to have the most amount of digital money. And when I quit the game, I had like a trillion something, several billions of dollars of just this fake currency, and so I, I don't know. I'm not as obsessed, probably cuz I'm a old man and I'm a dad and all that with a real job. And so, but either way, uh, I'm in Lihurst. So that's the location we want. The quality is zero, right? Cuz I'm looking at T3 meal soup and T5 planks, so quality is zero, so that means it's a 5.0 item. If you want changing this to one, it would be a 5.1 or a 4.1 or 6.1. 2, 3 and now four, but um, anyways, we're going to skip this tracker for now. This is how I pull, uh, this is how I get information when we do historical, which we'll skip right now. We'll skip this right here, skip this right here, and then I'm going to create an empty dictionary to put our current price data. So current price data equals and this, and then for each item in items of interest, So what we're doing is, we're going to do a for loop, first do this one, then do this one. So right now we're at T3 wheat soup or meal soup, I think it's called, which is the item name.
[7:19]And then we're going to build our URL to look like this right here. So what we're going to do is say the current price response equals requests.get. So the price URL, if you remember, is right here. And then we're adding, right? We're concatenating the item name, which right now is T3 meal soup, plus this string right here, cuz we want the locations and we want it in a JSON file. Equals location, which is Lihurst right now, if you remember, the qualities equals zero, so qualities plus zero, and now we've built a URL. So if the current price response. status code equal equals 200, 200 is the status code for requests, that means you've successfully, um, touched the website. Then I'm going to say market prices equals the current price respond.json, the zeroth index, which is the item name, and then the sell price minimum. So what did I just talk about?
[8:29]So let's say we do, this is what it looks like right here. So this is the zeroth index right here, the item bag for us is T3 meal soup. And then what we want is the sell price minimum. I don't want the buy price, right? That I'm not interested in, I'm looking in the sell price minimum, which is that first one right here. So right now we have a number, which I'll show you what it looks like. So we can print that out like this. So the current price, what we're doing an F formatted string, the current price market price for item name, which will start with T3 meal soup, and then go to T5 planks. In Lihurst is market prices, which is only one price, silver, but then I'm going to add that to a list. So item name, market prices, and then if for some reason you did the URL wrong, we print out failed to retrieve market data, and you can just try to improve your URL to make sure it's right. So let's print that out, right? So this is just the current price. Now, like I said, this is crowd-sourced data, so if you get a blank JSON, like I did for T3 meal soup, for some reason, there's nothing collected for the, um, for the T3 meal soup. So, um, and I'll show you that in the historical price. But for now, since we're printing out this string right here, the current market price for T3 meal soup in Lihurst is 1968 silver, and I just came from playing and that sounds exactly right. The current price for T5 planks in Lihurst is 486 silver, which sounds exactly right. And now I have this cool dictionary, um, that we can use for, um, something else I'll show you. All right, so that is historical price, or that is the current price, right? But you can also look at historical prices, but this doesn't work very well for me. Um, not sure why it's doing this, but, um, let me show you what this looks like. So print, uh, the result. So now what we're going to do is we are going to do the same thing, but now we're going to use this method right here, this history, the historical prices.
[11:15]And so now the URL instead of using this right here, this price URL, I'm just, I just made it explicit. So history plus the item name, so we're still in the four loop, so this will be T3 meal price plus the time scale equals 24, which is daily. And then same thing before, if the historical response is 200, then we're going to put that data, we're going to get that data from the JSON. Then for the D and data, I just want the location, if the location equals Lihurst or the location equals 1,000. So let's look at what that is real quick.
[11:59]So, um, item IDs can be here. Let's do the world text and let's look for let's look for 1,000. And there we go, that is Lihurst. So if it's Lihurst, or location 1,000, then we're going to take this result, right? Just like we did the current price data kind of, the result is up here, we're going to append the data to that. Same thing if we have a failed, it'll say failed to retrieve market data, and now we're going to print the result, right? So I took away these print, so right now we're just going to get historical data. All right, so now we have T3 meal soup.
[13:03]The item count, and we have a long JSON of the historical data for both of these things. And so I don't want quality data from a long time ago, so what I did was create a function that's going to take the time stamp. And for that, we're going to import datetime, from datetime, import datetime and time delta. And then the datetime do strip the time, this is the way that it comes in the JSON. So all we're doing is making this a object, datetime object. I need today's date, so datetime now. I want to see the last 30 days, so one month ago equals now minus the time delta of days 30. Right? So it's going to subtract that. If I added it, it'd be going forward and you wouldn't get any data. All right.
[14:03]So the result. append. So let's see what we do with this custom function, and now we can iterate over the data. And now for the item in result, so the result had that huge JSON right here. We want this item data equals data. We want the time stamp, right? So parse time stamp, it's going to come up here, right here, pass it the time stamp we just got. And then if it's less than or equal to one month ago, so everything within the 30 days, then we're going to, so if the filtered item data is within the 30 days,
[14:50]Then I want the location, the item ID, the quality, and then the data. Right? So now we're just going to have a smaller JSON, even though it doesn't look like it for the last 30 days of it.
[15:32]All right, so that's the two ways that you can use request to get the data. Um, like I said, I'm not going to show you a an application or anything, but here's kind of what I would probably use it for is, let's run this, and what I did was create a dash plotly, a plotly dash, website. This is kind of what it would look like.
[16:08]It's loading, it's pulling the data right now, and it's going to show the, um, prices. And because I'm recording and recording this video, it's taking longer than it should, but let's try to run it again.
[16:41]There we go. So T3 meal soup, the current market price for is 1968 silver. We can switch to planks and it switches to 486 silver. So that that's just one of the ways I wanted to do this, maybe when, um, Shane is fully up and running, I'm planning to have a market or a morning dashboard because before I go to the gym, I go on the line, but it would be nice to have everything here for me and one of them, maybe just the Albion Online market prices.
[17:15]So that's one way, that was using the current data. So same thing, it's all the same code, but this time I put it in a Pandas data frame, and then I made a dash website with an item drop down. That shows the value and then returns that market price. Um, I don't think I finished this one yet, but, um, I think I plotted out. Yeah, let let's see what this one looks like. And this one I'm using just plotly, um, not plotly dash, but here we go. So this is just to look at T5 planks at Lihurst. And for some reason, uh, the data only pros from March 13th to March 20th. I don't know why, but, um, you can kind of get a historical trend. So if you want to get really fancy with your your own personalized charts and everything, you can do that. And that's all I have. So I hope you use this and I hope it makes you a better all been online player. Uh, leave a comment if you want to meet each other in Albion, and thanks for watching. Goodbye, world.
