If this belongs in the other open source section, I'm happy to have it moved.

Reading carefully what Mark (Chogon) said, it looks to me that we'll have the code for the client, tools and server, and will have access to a data server with the data for MystOnline. By "data", I assume this means the ages that Cyan produced.

Unless we kept our data files we'll have to redownload the ages. Perhaps we won't have to, if we still have our MOUL files, but new players will have to redownload the ages. New fan created ages will also have to be downloaded. I assume that there will be something set up (an executeable file we download) that will initially download the client, the Cyan ages, and anything else we need.

I thought that our new Uru would work pretty much like UU. We had fan run shards with UU, and it worked. In my scenario, we would start with a bunch of unconnected shards, because that's the easiest way to manage Uru. It's a comfortable amount of stuff to manage. If one shard goes down, if one vault gets corrupted, the rest of the Uru shards will merrily go on. Sure, it's not all connected, but it worked in UU, and I thought it will work in our new Uru. It's the most conservative approach for running something that is dependent on volunteers. We have one success on fan run Uru - Until Uru. We'd have a second success with this new open source fan run version of Uru.

Then I thought about all those downloads. If we have to download the ages, that's a lot of data to push through fan run machines and network. It's not that people aren't technical and smart, but these Uru setups will done by people as a "labor of love". To download such big files from a fan run thing - that seems expensive.

Can we speculate how this will work? Will players download the ages from Cyan's data server (Cyan incurs the cost), and the rest of the files from the fan sites? Again, I assume we'll have an executeable file that runs the downloads, so we won't have to separately download one file and then another. Or, do you think that fan sites will grab the data files from the Cyan data server and we'll download the Cyan ages from the fan sites. And how will it work for new ages - what do you think?

My assumption is that we'll have to run the servers and address bandwidth issues in as cost effective way as possible. I can't see people paying a lot for our new online Uru, though of course I could be wrong. I can't even see them paying a monthly fee. Anyway, setting up a monthly fee payment and authorization system seems like a lot to do for a fan site, and not cheap to do either. Accepting donations seems reasonably easy (do the Paypal thing), but setting up a monthly payment and authorization process looks involved to me. Perhaps you could do a six month fee, and then give the player an authorization key, but that seems involved to me too, for a fan run sites and servers.

Now I'm less comfortable that it's all going to work out easily. I'd be overjoyed to be wrong. What do you think?

_________________
Retired forum mod, mszv - played as amarez in Uru
Personal blog - http://www.amarez.com, Personal twitter - http://www.twitter.com/amareze

Last edited by mszv on Fri Jan 02, 2009 6:31 pm; edited 1 time in total

Your assumptions are spot on. As for your worry: I'm confident we'll find a way to make this never be a problem. Should open source Uru ever become a smashing success where new people keep on coming in and data servers therefore keep having to upload content to them (I think it will be a while), we'll simply move to a more decentralized mechanism of distribution, like BitTorrent. It has worked for WoW; the only thing standing in the way of it working for Uru is that someone has to develop a patcher with a built-in BitTorrent client. Which would, in fact, not be a huge undertaking.

_________________
Sören Nils 'chucker' Kuklau; KI number: 221581
MYSTcommunity & MYSTlore
I write soeren says. You don't.

_________________
Denis 'amonre' Defreyne

My posts represent the views of myself, and not those of Cyan, unless otherwise stated.

Isn't BitTorrent problematic due to many ISPs either blocking or interferring with BT file transfers?

Wiki entry for ISPs that restrict BT.

johnsojc wrote:

Isn't BitTorrent problematic due to many ISPs either blocking or interferring with BT file transfers?

Wiki entry for ISPs that restrict BT.

Just like WoW does, we'd still provide an alternate non-BT channel. The patcher would automatically choose that if BT doesn't work satisfactorily.

(edit) Moreover, as that link points out, it is generally possible to work around throttling by using encryption. Given that Ur would ship its own BitTorrent client, we could use very strong encryption without having to worry about backwards compatibility at all.

_________________
Sören Nils 'chucker' Kuklau; KI number: 221581
MYSTcommunity & MYSTlore
I write soeren says. You don't.

chucker wrote:

johnsojc wrote:

Isn't BitTorrent problematic due to many ISPs either blocking or interferring with BT file transfers?

Wiki entry for ISPs that restrict BT.

Just like WoW does, we'd still provide an alternate non-BT channel. The patcher would automatically choose that if BT doesn't work satisfactorily.

Ooo goody, then we can throttle the connections to 10-20Kb/s and really annoy everyone

ddfreyne wrote:

Using BitTorrent, which was mentioned before, is a great suggestion. It should indeed be fairly easy to add support for BitTorrent-powered downloads to the client (depends on what the existing download implementation looks like), and it's definitely something I'll look into as soon as the Uru source is public.

Quite a few people in the community have dedicated servers with a lot of spare bandwidth; using BitTorrent would not only pretty much eliminate the bandwidth problem but would also make downloads faster. Win-win!

You have a good point here ddfreyne (about fans having a lot of spare band). I myself am planning on setting up a server on-release of code ASAP. Ive got a Dual Opteron 248 rig with over 1tb of HDD and a dedicated oc128. Best part? Because of the position im in i dont need to pay extra bandwidth costs. So anyone within about 400 miles of me can get insane download speeds. (Currently running several HalfLife 2 servers on this box). I am geo-located in the Wyoming/Colorado area with a fast pipe. So if we have a group starting a list of fan server locations/ips/etc PM me!

Just a side note as i see this as being somewhat part of this question. Do we have any estimates on how much data storage is needed for a shard? Looking at this in the perspective of how much band we would end up using at some point. (User data/etc).

How does one end up with a dedicated OC128 line that's sitting around and ready to run Uru?

_________________
Ro'to

Bittorrent can only be used as a second option. With different open source distributions it is important that each shard uses a dataserver that matches its vault data in order to prevent errors. Those who do not want to download from the dataserver can use the torrent but if there is a version mismatch the dataserver will redownload and overwrite the files anyway.

I estimate that there will be daily updates to the client data, most of them small (estimate based on my own output , which would be good for a weekly update). So a torrent would soon be obsolete.

eureka wrote:

Just a side note as i see this as being somewhat part of this question. Do we have any estimates on how much data storage is needed for a shard? Looking at this in the perspective of how much band we would end up using at some point. (User data/etc).

The size of my backup MOUL folder is 3.4 Gig. That is what you are going to need for a full dataserver. In theory you only need to have the data that was modified but as stated above you really want to replace any file that doesn't match.

_________________
D'Lanor (ɹǝʇunч puǝƃǝן uɐqɹn)

♪ KI# 29350 ♪

Last edited by D'Lanor on Sat Jan 03, 2009 1:57 pm; edited 1 time in total

D'Lanor wrote:

Bittorrent can only be used as a second option. With different open source distributions it is important that each shard uses a dataserver that matches its vault data in order to prevent errors. Those who do not want to download from the dataserver can use the torrent but if there is a version mismatch the dataserver will redownload and overwrite the files anyway.

I estimate that there will be daily updates to the client data, most of them small (estimate based on my own output , which would be good for a weekly update). So a torrent would soon be obsolete.

WoW uses multiple torrents to accommodate this. When you update it gets big patches, applies them, then goes back for the incrementals.

_________________
Can you withstand the gaze of the Eye of Eternity?

D'Lanor wrote:

Bittorrent can only be used as a second option. With different open source distributions it is important that each shard uses a dataserver that matches its vault data in order to prevent errors. Those who do not want to download from the dataserver can use the torrent but if there is a version mismatch the dataserver will redownload and overwrite the files anyway.

Huh? BitTorrent has no impact on the accuracy of the downloaded data. If anything, its checksums make the resulting download more (not less) likely to be valid.

Quote:

I estimate that there will be daily updates to the client data, most of them small (estimate based on my own output , which would be good for a weekly update). So a torrent would soon be obsolete.

…nor is creating a torrent something that can't be automated or requires a lot of time or computing power.

_________________
Sören Nils 'chucker' Kuklau; KI number: 221581
MYSTcommunity & MYSTlore
I write soeren says. You don't.

UU had patches but they were no longer used for MOUL. MOUL did only full downloads. Maintaining the patch tree is a big PIA. But anyone who is prepared to reinstate them would be more than welcome.

_________________
D'Lanor (ɹǝʇunч puǝƃǝן uɐqɹn)

♪ KI# 29350 ♪

chucker wrote:

D'Lanor wrote:

Bittorrent can only be used as a second option. With different open source distributions it is important that each shard uses a dataserver that matches its vault data in order to prevent errors. Those who do not want to download from the dataserver can use the torrent but if there is a version mismatch the dataserver will redownload and overwrite the files anyway.

Huh? BitTorrent has no impact on the accuracy of the downloaded data. If anything, its checksums make the resulting download more (not less) likely to be valid.

Quote:

I estimate that there will be daily updates to the client data, most of them small (estimate based on my own output , which would be good for a weekly update). So a torrent would soon be obsolete.

…nor is creating a torrent something that can't be automated or requires a lot of time or computing power.

Oh wait. I missed the part of integrating bittorrent into the client. I thought they would be separate downloads.

Anyway, I am still not in favor of bittorrent because for me torrents are much slower than the MOUL downloads were. I think dedicated dataservers are the way to go, although I would not mind if segmented downloading was integrated into the client to make optimal use of the dataserver cluster.

_________________
D'Lanor (ɹǝʇunч puǝƃǝן uɐqɹn)

♪ KI# 29350 ♪

_________________
Denis 'amonre' Defreyne

My posts represent the views of myself, and not those of Cyan, unless otherwise stated.

Thanks all - I feel better!

I did think about Bit Torrent, after I posted. My experience with Torrents has been mixed - always reliable, but sometimes slow. I do understand that it has to do with the connection of the seeders.

I'm still thinking we'll have a set of unconnected shards to start with, and then perhaps more interconnections later.

_________________
Retired forum mod, mszv - played as amarez in Uru
Personal blog - http://www.amarez.com, Personal twitter - http://www.twitter.com/amareze