• Want to support a good cause? Visit https://blacklivesmatters.carrd.co/ to see how you can help the Black Lives Matter movement, even without donating.
  • No paparazzi clickbait here! Just a good old fashioned feature with exactly zero gossip about celebrities' love lives. Honest.Check out our spotlight of Rhema's storylocke, As Above, So Below.
  • One of our new features includes receiving a message on Discord when you have alerts. Find out how, here!
  • Have you seen our Stream tab? It let's you peek in and chat with our forum streamers on-site, from both Twitch AND Picarto! You can even view multiple streams at once. You can submit your own channel via Streams > Submit Channel.
  • If you're a fan of giving and recieving constructive criticism check out this resource about Critique Level Tags!
  • The results are in for the Nuzlocke Forums Extravaganza! Congratulations to everyone that placed this year!

  • Episode Six of The Writer's Locke podcast is now available for stream or download! This episode continues our exploration of the new stories, and in our discussion segment we tackle more of the planning process!
  • New here and still figuring out the site? Check out the New User Guide and FAQ for some help!
  • Trying to figure out how the different forums have changed? This thread is the place for you.

NOTICE: Upcoming Server Migration and Site Host Changes

Fullmental

*Fódlan Winds intensifies*
Administrator
Moderator
Tech Team
🎇Contributor
Writer
Team Omega
Pokédex No.
2
Caught
May 13, 2019
Messages
391
Location
Fula City
Nature
Calm
Pronouns
He/Him
Pokémon Type
Rock, Clever
Pokédex Entry
This Pokemon can sometimes be found meticulously editing and re-editing it's own content, never satisfied with the results.
Hi everyone, we have some very important news to share with you today: In early May 2020, we will be switching hosting providers.

This does not mean we are moving from Xenforo, and our URL will not change either. All of the changes will occur behind the scenes.


What Went Wrong

As you know, last year we made a major change in our switch from Tapatalk to Xenforo. While we were and still are largely happy with the forum software itself, the forum software was not the only part of our large scale change in 2019. We also had to find a place to host our shiny new site. That is where our current issues begin...

I will be blunt: When we moved from Tapatalk to Xenforo, we were ill-equipped and ill-informed to choose what sort of hosting service we needed. We had no idea what we were doing, we just knew we needed to get away from Tapatalk as quickly as possible. When it came time to start moving things over, we began with a shared hosting platform, thinking that - since we're just creating a forum - we couldn't possibly need a lot of resources, right? Unfortunately, that was not the case. So while we forged ahead with a three year service plan for our shared hosting service, we were unknowingly digging a proverbial hole for ourselves that would be difficult and expensive to climb out of.

The following section goes into the various issues we've encountered and steps we've taken to try and resolve them throughout the past year. If you're not interested in some of the more technical details, feel free to skip ahead to the summary section.


How it Went Wrong - Strikes One and Two

When we first began testing on our beta site, we found that it simply couldn't run on the shared hosting plan we selected without some serious slowdowns and limitations. That was a bad sign, considering we hadn't even imported or posted anything yet.

As it became clear that a shared hosting plan wasn't going to fit our needs, we went to our hosting provider to try and work out an upgrade path. We went back and forth for a week or so, dealing with some hassle as our first choice for an upgrade (Windows managed VPS) wound up being incompatible with some services. We had to fight again to get a change from a Windows host to a Linux host without losing our existing investment into the shared plan. The support team informed us that they generally did not allow refunds on partially used hosting plans, but they would make a one-time exception for our needs. Warning sign #1.

We wound up on a VPS hosting plan with 1 virtual CPU and 1GB of RAM. However, even with this managed Linux VPS, we still ran into performance issues all over the place. You may remember the first several months with Xenforo - we had a lot of sudden server crashes and site loading was extremely slow. We had to put our immediate plans to move over all tapatalk threads on hold, after our import of FCR caused massive CPU and RAM spikes. As we learned more about the software and services we were using, it became clear that, while our site was very poorly optimized at the time, our entry level VPS plan was far too weak to properly support our large forum database, even after we could clean everything up. After all of that work to get moved over to our plan, we had to bump up our plan again to a managed 2 vCPU, 4GB RAM service. Again, we had issues moving over quickly and did not get full value of our previous services in the upgrade from our hosting team. Warning Sign #2.

It was also about this time that we formed the Nuzlocke Forums Tech Team, in the hopes that a few of our more technical staff could work together to optimize the software, and learn to properly maintain the forum ourselves. It was our hope that, by the end of the 3 year hosting plan, we could switch to an unmanaged service. This meant that we would be in complete control of the entire server, from the software level down to the kernel level. It also meant that we would have no safety net if we ran into a problem that we couldn't resolve. It was (and remains) a scary proposition to a team that, until very recently, had never managed a server entirely on our own before.

In recent months, we've finally gotten our forum software running smoothly. Aside from planned reboots and maintenance, the server has been nice and stable, and the performance issues have largely disappeared. This is in part due to the use of Cloudflare as a CDN, as well as due to a major change in our database configuration. It's also in part due to some firewall rules we've set up to block abusive web traffic, which was previously bringing our server to its knees with constant database queries from bots and scrapers. Additionally, we identified several forum add-ons that were causing performance issues, either due to an incompatibility with other add-ons, or due to poor coding practices.


How it Went Wrong - Strike Three

Despite these performance improvements and resource optimizations, our system remains starved for memory in particular. Our CPU and disk usage is well under control, but such a large forum database of 700,000+ posts and our constantly expanding media gallery needs a large amount of memory to function properly. Without getting too technical, our database is sustaining itself on about half of the memory it should be given, and our system as a whole is running extremely starved of system memory as well. In all, we estimate that, with our current VPS and backend setup, we would need to double our system memory from 4GB to 8GB to allow optimal cache and buffer sizes across all of the back-end services.

We've evaluated our options, none of which are ideal. Because we paid for three years of hosting in advance, any upgrades with our current host would *also* need to be paid ahead to 2022. In order to move to a VPS with 8GB of RAM, we would have to nearly *double* our service cost, if we stuck to a managed plan - this was out of the question.

What we *could* do was try an unmanaged plan. We again reached out to our hosting provider to try and find a reasonable upgrade path. Unfortunately, they were unwilling to guarantee we would get a proper prorated refund for the amount of our current plan and we had to pressure them several times to even get numbers. The entire negotiation phase took more than a week, with very slow, very uninformed personnel throughout most of the process. Despite our best efforts, we gave up on trying to work with them.

In the end our host declined to help us until we floated the idea of spending hundreds of dollars more for the upgrade. What's worse, they also tried to hide these upgrade costs from us until we committed to the change. This was the last straw. Our hosting provider always was difficult to deal with, but now they are actively hindering our ability to run the site as we feel is necessary. We no longer want to give a single cent more to our hosting provider, which leads us to alternative options...


How we Plan to Fix It

If we were to stay on our current provider without making an upgrade, we would need to cut back on current features in order to remain stable long-term. This means disabling add-ons the community may see as highly useful or even essential, such as Discord integration, thread and post alert overhauls, and email notifications, as well as indefinitely postponing further imports of the Tapatalk archive. We do not want to do this as it will negatively impact everyone who enjoys the Nuzlocke Community at large. So we can't stay where we are, and we can't pay our hosting provider for the upgrade we want in good faith. What else can we do?

While we will not be able to recover the money we already spent on our current host, we can stop paying them more in the form of our yearly license and domain fees by simply going somewhere else. If we were to stay with our current provider, we would have to pay a relatively large sum to them in those fees and our needed upgrade.

If we instead take this money - which we would have needed to spend anyway - we can go pretty much anywhere else. Specifically, though, we feel it's best to go to a hosting provider that gives us the option to upgrade and downgrade at will, and tweak our resource setup to better suit our needs. This would prevent us from falling into the trap of a long-term service plan - one which may or may not meet our needs 2-3 years down the line. This move will essentially achieve the same thing as our planned "upgrade" with our current provider, but if done right we can be on a strictly usage-based plan, billed monthly. No strings attached.

Right now, we are evaluating two providers that offer these sort of scalable, node-based plans. As an added bonus, we can also separate our services out so more powerful tools (such as our database) can run with their own dedicated resources. Some of you who are more technically inclined may be able to guess which providers I'm talking about here, but right now we are keeping these choices internal. We will make a formal announcement as to which host we are moving to at a later date, and for now nothing is set in stone.


How This Affects Our Budget and Our Community

As you know, we are entirely donation funded. We don't run ads, we don't sell data or services, and we operate entirely on volunteer work. Last year, we ran a fundraiser to pay for the first three years of hosting, and now we're throwing away two of those years. Is this fair to those that contributed to that fundraiser, giving either their time or money to secure our future?

To be honest, this is the question what we have struggled with the most. We considered polling the community, but after much deliberation we've ultimately decided that we would be doing everyone a disservice by not making the change. It would be just as wasteful and problematic to stay where we are as it would be to move providers, but we would be in a worse situation at the end of the change if we were to stay.

Because of this, we decided to move ahead without soliciting opinions on the change. Still, we want to briefly touch on the budget situation and explain what is happening as a result of this.

If we were to stay with our current provider, we would owe them anywhere between $270 and $290 to do the VPS upgrade (we couldn't get an actual value from the billing support team), and $180 per year to maintain our backend licenses with two years remaining. Assuming these prices remain the same and our licenses don't increase (which they might), we would be in for $635 in additional funds to our current provider.

If we were to move providers, that is when things get interesting. I'll explain the specifics in detail further down, but for now we are looking at plans from both providers that will allow us to split our hosting duties between two cheaper "node" plans. These two nodes will cost us $25/month to start, saving $15 for every month we remain on that plan compared to our proposed upgrade with our current provider. We can't guarantee that is enough especially as our tapatalk archives begin getting imported again, but we don't have to - we can change the configuration as needed without paying a fee or waiting until the end of a billing period.

If we were to stay on this plan for two years though, that $15/mo savings would add up to $360 - actually saving us $35 over staying with our current provider. Realistically speaking, we expect that we will need to scale up $5-10 a month after about 6 months, where we can hopefully stay for the foreseeable future. This represents a 6-month cost of $150, followed by a 18-month cost of anywhere between $540 and $630 for a total of $690-780. This is less than the cost of a comparable plan using one node ($960), but slightly to moderately more than the estimated costs to stay at our current provider (~$675). Given the upsides, we are confident that this is the best path forward for the site, even if it is slightly more expensive.

Remember that these are all our best estimates. The values could and probably will change. Based on all of the information we've gathered since we started running this site, we have reason to believe that, in addition to our flexibility, this will be a cheaper option long-term than any traditional VPS hosted plan. Therefore, it is a better use of community-donated funds than staying where we are.



I do want to explain to those interested how we got to the numbers we did just now for the new plans. Going into the reasoning behind these specific costs with our potential new providers does require a little bit of technical jargon, so I've hidden the details in the spoiler below. If you don't want to read it, feel free to move right along.

Our potential new providers offer plans that are highly scalable. Getting the same 4vCPU, 8GB VPS that our current provider offers would cost us $40 a month. This would cost us $960 over the same 2 year term, an increase of $325. However, we are not strictly limited to this plan, and our newfound flexibility can help to close the gap.

We've done our research on this and think we can split our nodes out to isolate the RAM hungry database engine, allowing us to go smaller on the main forum service (which is extremely lightweight). Specifically, we really don't need 4 vCPUs for our site. We are barely using the 2 vCPUs we have now. To demonstrate this, I do have some performance metrics to share.

The chart below shows our CPU usage over the past year. 1 CPU core represents 100% usage on this graph, so 200% usage is equal to full load on our current 2vCPU hosting plan. This graph is not zoomed in enough to see daily or weekly load averages which will see higher spikes, but generally speaking we need enough overhead to cover our system load during peak times. In general, our system remains under 50% load for any given 15 minute average.

1583552915466.png

You can see that, when we completed our database and CDN optimizations in mid November and early January, our CPU usage dropped significantly - the graph remains under 10% utilization, or 5% per core for average load. Our system could sustain itself with 1vCPU easily, or possibly even less. As a result, we really don't need to worry about CPU cores when moving.

The resource we are in short supply of is system RAM. We have always been just barely functioning, but as traffic has increased and our database size has grown, it has gotten even worse over time.

1583552943239.png

Now, this chart takes a bit of explaining. System monitoring was only enabled in mid-July, so we don't have any data on how our resources fared before our upgrade to 4GB of RAM. What we want to look at here is the top of the "unused" fill in neon green, and how it compares to the dark blue fill below it. The top of the neon green fill represents represents our total system RAM and the top of the blue fill shows how much memory is in use, either as cache for our database and OS, or memory in active use by apps and services.

You can see that the total system memory in use (the top of blue fill) is always extremely close to the available memory (the top of the neon green fill) from mid-July to November. This was when we were experiencing poor system responsiveness and frequent crashes. In response, the Tech Team made a change to reduce system cache and trim a few unneeded services in order to improve stability. This worked for a bit, but the site was still slow and the database was frequently locking up.

When we switched to the new database engine, we had to allocate more of that unused system memory back to keep it running. The new database was much faster, but needed more RAM to stay healthy. Again, we were pushing 90% RAM usage, which caused more out of memory errors and site crashes.

Finally, in December we turned on system swap and gave everything an additional 4GB of "memory" by using the hard drive as temporary storage when needed. This is extremely slow and taxing on the drive, so it cannot be maintained indefinitely. Still, this is what has been keeping us afloat the past several months while we've explored alternative options. You can see that, even with our previous cuts in system cache, our system wants to pull about 6GB RAM. If we had cache levels and buffer sizes where we wanted them, it'd be closer to 7GB. This is why we are in a bad spot - we're unable to make significant feature changes or import more large amounts of threads due to these memory limitations.

That in mind, our proposed hosting provider does not use a "traditional" hosting setup. It is more barebones, requiring fewer resources just to keep the lights on, so to speak. We will be able to further drop the system cache and more carefully control random allocation requests by splitting out the needy services into their own nodes. As a result, we've come to the conclusions that, with these system services gone, we would only require 5GB RAM across all nodes instead of 6-8. We still think this will be a little small, but we will start there and see how the system behaves. We can always give a service more resources if necessary, with no penalty for the change.


When is This Happening?

While the planned move date is not entirely set in stone, we do have a hard deadline to meet. Our next year's licensing fees are due on May 4, 2020. We do not plan to pay them, so we must be fully migrated to the new hosting provider before then. The Tech Team is currently conducting some basic testing on both potential future hosts. This includes evaluating and testing our system requirements on various system configurations, documenting and perfecting a migration procedure using our existing forum backups, and conducting some basic load testing to compare with our current activity and ensure our proposed node distribution will meet our needs in the short term. This testing is being funded by the staff and is expected to run less than $20.

When we have selected our next hosting provider and confirmed our migration steps, we will review our final decisions and decide on a date to make the move. When the move occurs, we will set the forum to maintenance mode in order to prevent any posts from being made during the migration process. The backup and system migration is expected to take anywhere between 12 and 24 hours, and the DNS propagation will take anywhere from a few hours to a few days depending on your ISP. We realize that some people having no access to the site for up to three days is not ideal, so we will look into any options we have for reducing that, if possible.


Summary

Got a major case of TMI? Did your eyes glaze over? We understand - this is a lot of information that most people never have to think about, much less make decisions on.

Here's the tl;dr:

We've been having issues with our current hosting provider ever since we made our initial move. We've tried to work through the issues by optimizing the site and jumping around on our hosting plan, but we have reached the point where our hosting provider's policies and behaviors are hindering us, and the support teams are actively hiding information we need to make informed decisions moving forward. Rather than continue to remain with them and pay ongoing licensing and upgrade fees, we have decided to forfeit our remaining two years and move to another provider that will be more flexible. Our expectation is that this flexibility will absorb the impact of this move by helping to save the site money over the longer term without locking us into a specific service plan.

The move will take place prior to May 4, 2020, at a date which will be decided on in the coming weeks. You may not be able to access the site for a few hours to a few days, but this will be outside our control. Please bear with us as we handle this move as efficiently as possible.


FAQ

Who is your current hosting provider? Why do you refuse to name them?
We would rather not publicly shame our current provider while we're still dependent on them. While we're not happy with our provider and have grown increasingly frustrated with them over the past few months, the sad truth is we would have run into this sort of issue with almost all "traditional" hosting providers. Very few offer a generous refund policy or allow frequent plan changes, so our own lack of knowledge was a major factor in getting ourselves into this mess in the first place. Our experience is also not necessarily reflective of the provider's other services, such as dedicated hosting or shared hosting for more common sites like those driven by WordPress. In fact, our current hosting provider is one of the most well-rated today in terms of value and features. Your mileage may vary.

Which providers are you looking at now?
You can probably make an educated guess based on how they handle billing, but we're going to make another announcement later with the one provider we selected. We would prefer that community discussion did not influence the final decision. Instead, we would like the Tech Team to make this choice based on our needs.

Will we lose any site features or posts as a result of the change?
No. We will take a site backup immediately prior to the migration, so the site will be exactly the same on the front end. Everything that is changing is behind the scenes.

What does this mean for donators, and who is funding the extra cost associated with the move?
While there is nothing we can do to recover the money already paid to our hosting provider, we do not plan to ask the community to pay the short-term costs associated with this move. A few staff members have decided to contribute towards the migration and the increased hosting costs for the remaining two years. Any fundraising or donations in the near future will go directly to our Xenforo license and Add-on costs, as well as towards the development or purchase of new/improved site features.

When is the rest of the Tapatalk archive being imported, anyway?
We did mention in this announcement that the Tapatalk archive import process had to be put on hold several times since our first move to Xenforo. Currently, the team is on hold again, waiting until the migration before doing any additional work. After that, work will resume as time and resources permit. The import process is run by a small subset of volunteers, so they will work as time allows. We do not currently have an estimate for when it will be 100% complete
 

Rumors

Or so they say...
Writer
Pokédex No.
258
Caught
Jul 1, 2019
Messages
245
Pronouns
male
Pokémon Type
Fire
Pokédex Entry
A Pokémon shrouded in hearsay and urban legend, spoken of only in whispered tales and hushed gossip.
Just throwing a potential concern out here: is there any contract with the current provider? If so, are there any sort of paragraphs, clauses, sections, or anything of the like discussing non-payment or breach of contract? If so, or if uncertain if there is so, it'd be a very good idea to review that before making a commitment. Contracts can be really nasty about stuff like that and potentially open us up to who knows what, such as requirement remaining balance for the other two years paid in full up front.
 

Fullmental

*Fódlan Winds intensifies*
Administrator
Moderator
Tech Team
🎇Contributor
Writer
Team Omega
Pokédex No.
2
Caught
May 13, 2019
Messages
391
Location
Fula City
Nature
Calm
Pronouns
He/Him
Pokémon Type
Rock, Clever
Pokédex Entry
This Pokemon can sometimes be found meticulously editing and re-editing it's own content, never satisfied with the results.
  • Thread Starter Thread Starter
  • #3
Just throwing a potential concern out here: is there any contract with the current provider? If so, are there any sort of paragraphs, clauses, sections, or anything of the like discussing non-payment or breach of contract? If so, or if uncertain if there is so, it'd be a very good idea to review that before making a commitment. Contracts can be really nasty about stuff like that and potentially open us up to who knows what, such as requirement remaining balance for the other two years paid in full up front.
Hey there - thanks for the note! Yes, we did look into this for those same reasons. Thankfully, the services that will cost us yearly only have yearly commitments - all invoices have been paid upfront for the required terms to meet the hosting provider's condition of sale. These are things like our software licenses and domain registration. For the domain, we can easily transfer that.

The reason why we called these renewals "required" in the post is because removing them without moving to another host will essentially wipe out our current server's configuration, and we would have to start from scratch anyway without that framework in place. It is not because we have a contract that states we must re-up these licenses. Our host's terms allow us to immediately cancel all services with no fee - we will just not receive a refund for remaining time that we already paid for (a few days on the yearly commitments, 2 years on the VPS itself). We just need to submit that cancellation request prior to the "renewal" of these licenses.
 
Last edited:

Skarmizard

The metal dragon bird
Writer
Screenshotter
Pokédex No.
1002
Caught
Nov 12, 2019
Messages
555
Location
Texas
Nature
Quiet
Pronouns
he/him
Pokémon Type
Dragon, Steel
Pokédex Entry
This Skarmory/Charizard fusion is very rare and tends to be quiet and shy until befriended. Also a huge fan of Smash Bros. and writing.
I'm beyond confused...so if I'm understanding correctly, the site itself is going to be the same, it's not a move like when we moved from the old forum? Is anything going to be different on our end?
 

Fullmental

*Fódlan Winds intensifies*
Administrator
Moderator
Tech Team
🎇Contributor
Writer
Team Omega
Pokédex No.
2
Caught
May 13, 2019
Messages
391
Location
Fula City
Nature
Calm
Pronouns
He/Him
Pokémon Type
Rock, Clever
Pokédex Entry
This Pokemon can sometimes be found meticulously editing and re-editing it's own content, never satisfied with the results.
  • Thread Starter Thread Starter
  • #5
I'm beyond confused...so if I'm understanding correctly, the site itself is going to be the same, it's not a move like when we moved from the old forum? Is anything going to be different on our end?
Hey there! That's correct - nothing will change on the site itself. It's all going to be happening behind the scenes.

The reason we are taking the time to explain this in great detail is because it impacts us financially. Since we're run by donations, the community deserves to know what we are doing and why. There will also be some downtime during the switch.
 

PurpleEyedGuma

21st Century Schizoid Mon
Writer
Team Omega
Pokédex No.
883
Caught
Oct 9, 2019
Messages
464
Pronouns
They / Them
Oh, I thought for a second we were switching away from Xenforo. Thank you for clarifying.

EDIT: Xenforo seems like our "site host" already. What exactly will change? Will all our previous posts be intact, or will we switch to an entirely new Xenforo site?
 

Fullmental

*Fódlan Winds intensifies*
Administrator
Moderator
Tech Team
🎇Contributor
Writer
Team Omega
Pokédex No.
2
Caught
May 13, 2019
Messages
391
Location
Fula City
Nature
Calm
Pronouns
He/Him
Pokémon Type
Rock, Clever
Pokédex Entry
This Pokemon can sometimes be found meticulously editing and re-editing it's own content, never satisfied with the results.
  • Thread Starter Thread Starter
  • #7
Oh, I thought for a second we were switching away from Xenforo. Thank you for clarifying.

EDIT: Xenforo seems like our "site host" already. What exactly will change? Will all our previous posts be intact, or will we switch to an entirely new Xenforo site?
Hey There. Xenforo isn't a site host, it's a piece of software that we use. Site hosts are companies like Bluehost, A2 Hosting, GoDaddy, and HostGator which offer a platform for you to run your site. Think of them as the computer or phone you use to check this site or Discord. When you need to get a new one, there's a process for moving everything over to your new device but it is still your data, and your contacts when you're done. That's sort of what we're doing. We're picking up our data, and moving it to another device, but the data itself isn't changing.

I do realize this is a bit confusing. Tapatalk is both a platform and a host, meaning the device and the app was one service. That is not the case with Xenforo. We own the software outright, and can do whatever we want with it. Xenforo doesn't host the forums for us, we do. It's very different from how things were before.
 
Last edited:

Users Who Are Viewing This Thread (Users: 0, Guests: 1)

Top