Shop Mobile More Submit  Join Login

Similar Deviations
Subject to change due to work in progress
The Best Works of the World Wide Web (W3) or Internet

(Right Click) Open Image in New Tab (use Magnify Glass tool) to see bigger pictures, text and author/credit

List of Star Wars spacecraft…
List of Star Wars starfighters…


Fighters and Bombers
Add a Comment:
No comments have been added yet.

Install notes for Ubuntu/Linux

Journal Entry: Tue Oct 2, 2012, 12:05 PM
Install notes for the Open-sources digital painting software I use on Ubuntu/Linux (12.04 ) ;
to install the recent version of them, in a terminal copy this line by line :

Mypaint 1+Git:
sudo add-apt-repository ppa:achadwick/mypaint-testing
sudo apt-get update
sudo apt-get install mypaint

Krita 2.5.2 :
sudo add-apt-repository ppa:kubuntu-ppa/backports
sudo apt-get update
sudo apt-get install krita

Gimp 2.8 and G'Mic :
sudo add-apt-repository ppa:otto-kesselgulasch/gimp
sudo apt-get update
sudo apt-get install gimp gimp-gmic

[update] more detailed infos ; on my blog [here]

Add a Comment:
No comments have been added yet.

Star Wars: A New Hope

Thu Nov 20, 2014, 7:41 PM
Img-00a by techgnotic

Who’s the most important person now in the Star Wars franchise? JJ Abrams? Think again!

It has begun. Below you will find the trailer to the beginning chapter of the next Star Wars sequels, Star Wars VII: The Force Awakens,a new trilogy for a new generation. One well crafted 88 second trailer later, and it's just what we were hoping for. X-Wings, Tie Fighters, brand new Light Saber designs, the Millenium Falcon and just enough intriguing character moments to have us all guessing at plot points until the next one arrives. The style of camera work is also very interesting, a bit cinema verite, as we open on a character obviously on the run. We will have a full breakdown of the trailer coming next week. Our global Star Wars culture stirs in its sleep. George Lucas is whispering, “Wake up…” — for he has more life lessons for the youth. He continues on in his role as our children’s modern Walt Disney.

This second batch of Star Wars movies will hopefully fare better with the fans than the near-disastrous rollout of The Phantom Menace in 1999. The Star Wars community was split in two by the prequel trilogy. While on one hand it brought in a whole new younger audience to the franchize, the older fans who had waited so many years for these films were devastated that they were skewed to such a young audience.

But there was something else missing, one might say the vital ingredients that made the original franchize so great.

There was no hope that we’d ever see Francis Ford Coppola, writer and director of the fantastic Godfather Trilogy as well as Apocalypse Now, work on the scripts as he had done on A New Hope. There was also no chance that Gary Kurtz, the original trilogies producer would be back either. Gary was just as involved with Star Wars as George Lucas was. He kept George on track on both development and production through American Graffiti and their early desire to do a Flash Gordon adaptation, the latter project evolved into Star Wars.

We can relax because J.J. Abrams, who so successfully resuscitated the Star Trek franchise with his reboot, is the director this time out.

No worries, right? So who’s left?

Lawrence Kasdan. The man who co-wrote Empire Strikes Back and Return Of The Jedi. The man credited with breathing life into Luke, Leia and Han, then taking them on a slightly darker path. Kasdan was the man who also helped bring Indiana Jones to the screen, co-writing Raiders Of The Lost Ark! Many fans, with serious Star Wars knowledge, were relieved when the first video journal for Star Wars had both JJ Abrams and Lawrence Kasdan in it. A hopeful sign that JJ wouldn’t have final say on the script, something many Star Trek fans had wished someone else had on that reboot. Lawrence will be around for a while, his name already linked with the further sequels and spin-offs planned by Disney for Star Wars.

So as the Jedi nation grows in its multitudes and the 2015 Star Wars Celebration Convention, scheduled for April 16-19 in Anaheim is projected to shatter attendance records, there remains little evidence of the Force ever having being asleep in these years after the third “prequel.”

Lawrence, our faith is with you.

No pressure.

Your Thoughts

  1. Which Star Wars characters are best suited to be featured in their own spin-off films?
  2. Do you hope Lando makes a surprise appearance?
  3. What are your hopes for the new series of Star Wars films?
  4. How would you like to see Han Solo and Princess Leia's relationship play out?
  5. What is your favorite Star Wars Fan Art or Fan Fiction?

Who’s the most important person now in the Star Wars franchise? JJ Abrams? It has begun. Below you will find the trailer to the beginning chapter of the next Star Wars sequels, Star Wars VII: The Force Awakens, a new trilogy for a new generation. One well crafted 88 second trailer later, and it's just what we were hoping for. X-Wings, Tie Fighters, brand new Light Saber designs, the Millenium Falcon and just enough intriguing character moments to have us all guessing at plot points until the next one arrives.

Author/Curator: DeevElliott
Designer: seoul-child 

For more articles like this, visit depthRADIUS.
Want to submit any ideas, suggestions, collections, or an existing work for consideration for the Today page? We'd love to look at it. Email us at
Add a Comment:
No comments have been added yet.

I do them, yes! It's one of my favorite pastimes. ^^ For those interested, here's a general list of things (anime, shows, games, whatever) I'm familiar enough with to RP the characters, or some of the characters.

Notice: Any animes listed exclude manga canon, since I don't read it much.

Old Journal
- Avatar: The Last Airbender (no LoK)
- Attack on Titan
- Soul Eater
- Harry Potter
- SOME Pokémon (games only, to an extent; anthro Pokés or humans)
- Teen Titans (been YEARS since I watched, though)
- Death Note
- Another (set BEFORE all the deaths, obviously; same for Angel Beats and Madoka Magica)
- Angel Beats
- Madoka Magica

Things I forgot in the old journal:
- Mass Effect (no using Shephard on either end)
- Skyrim
- DBZ (new!)
- RWBY (new!)
- A million other things slipping my mind currently, etc.

Things I just recently became a fan of:
- Psycho-Pass
- Kid Icarus: Uprising
- Guardians of the Galaxy
- The Devil Is A Part-Timer (new!)

There's probably way more stuff I'm not thinking about just yet, but this is just stuff in general I know I can RP characters from. I'll also play with or as OCs as well, so there's that. I also will not do anything sexually explicit, but can hint at darker themes. Almost any bondage set up is fine (including none!), along with however intense it can get for the 'lee. I won't refuse to play the ticklee if asked, but generally play tickler more often. 

I'll play via notes or Skype, but if you have an alternate means of communicating (I won't RP in the comments) then we can work something out. Note me for my Skype name or to start an RP!

Add a Comment:
No comments have been added yet.


Wed Jan 18, 2012, 1:52 AM

October something, something.
Today captain declared as "REMEMBERANCES OF THE PAST DAY".

The day begun with me getting smacked with a broken laptop and yells "YOU'VE GOT MAIL!".
I tried to protest that laptops weren't attacking people in the mornings back in the past. To this, Captain declared that in fact they didn't have to, because users were so addicted to reading their daily mail facts that every morning they woke up smacking their heads on their laptops.

Then my head was treated to a barrage of empty, metal SPAM cans with words "SPAM MAIL! DODGE THE SPAM! USE FILTERS!".
With these words, captain gave me two old tennis rackets. The tennis rackets had yellow duck-tape stickers on them with words "DELETE" and "BLOCK EMAIL ADDRESS".

The spam can attack didn't end there, Pilot's job for today was hiding behind building ruins and throwing the spam cans at me at random intervals with yells of "INCREASE YOUR GENITAL SIZE!", "RUSSIAN BRIDE LOVES YOU" and "I AM NIGERIAN PRINCE WITH 56 BILLION DOLLAR OFFER FOR U".

Captain then gave me a filthy, blue sash that said "INTERNET EXPLORER 37.0" and demanded me to fetch foods from EBAY.
The old shopping mall where we found supplies yesterday now bore a massive spray painted sign "EBAY" on it. I still have no idea how they created this masterpiece of utterly pointless vandalism. Short of captain holding Pilot upside down and Pilot spray-painting the letters I coudn't think of anything.

Upon opening the mall's doors, I fell straight through the floor. Damn mold must have eaten through the old floor, I thought... but then I heard captain's comments "INTERNET EXPLORER HAS CRASHED" and it became clear to me that the floor was somehow sabotaged.
I decided to be much more careful from there on, knowing that there would probably be more traps that were meant to crash internet explorer.
"POPUPS!" Captain's voice bellowed from above, as cardboard boxes rained on me. One of them with the words "XXX GIRLS" hit me square on the head.

I woke with Pilot hovering over me, wearing a greenish, moldy sash with words "NETSCAPE NAVIGATOR".
"YOU WILL BE REPLACED!" Pilot's arms flailed back and forth in front of my face like a crazy dance.

I didn't like the sound of that.
I saw the worm monster being slowly lowered into the hole, red stripes were painted on it. The hungry beast emitted screeching noises that echoed throughout the building. At this rate, its friends would be coming soon.
I reached for my rifle strap, finding nothing. Great, they must have taken it while I slept.
Pilot stared at me from the corner, then at the worm monster with red stripes, and then made a jump for the door.
I followed, picking up speed.
Pilot thew another SPAM can at me, and I barely dodged it.
Netscape, I mean Pilot.. was already halfway up the stairs.
Suddenly, a sack of SPAM cans was emptied from above, hundreds of cans coming down the stairs making a horrid rattling noise.
Pilot pulled two rackets that were tied to his back, slapping the cans out of the way.
I had no such defenses, as I left the rackets back in the first room, and thus the cans rained on my head, derailing my steps and slowing me down.
Firefox screeched and crooned from behind, urging me to run faster. It sounded like a mad pigeon amplified through a megaphone.
Pilot jumped across several steps. I think he did serious gymnastics or something back in the army, damn my office life.
Just as I reached the door, captain appeared in front of me, blocking the way to freedom.
A large black board connected with my facemask as the metal door bearing it snapped shut right in front of me.
There were chalk scribbles on the board spelling a 4 letter word" S O P A " and below it "THIS ENTRY WAS CENSORED BY U.S GOVERNMENT".
I desperately clawed against the board feeling surely doomed today, just like the 2012 internet users.

Add a Comment:
No comments have been added yet.

Tip of the Day: Staying Successful

Journal Entry: Thu Apr 5, 2012, 9:50 AM
One of the biggest obstacles a person can face is to stay successful after being successful. Stay hungry! Let's go!!

Sending positive vibes to you all!

Add a Comment:
No comments have been added yet.

Charles Snippy: COFFEES

Sun Feb 26, 2012, 9:48 AM

ENTRY: 412:

This evening, I returned to into the City to fill out the reports about my first tour out into the Dead Zone.
Hopefully, they won't suddenly shut down the D.Z.T.R. Department and there will be more tours for me to "guide".

Today... was different.
A buzzing, hovering Hunter-Drone met me in the Airlock.
Must be a new thing.

Ah I see,
As I'm forever unable to access ANNET, someone up top must have programmed this contraption to follow me around, shouting things at me.

Thank you, whoever you are.
I never felt more ridiculous.

To top things off, I lost my coffee.

Add a Comment:
No comments have been added yet.


Thu Mar 1, 2012, 9:51 AM

DeLays, wE WeRe DeLaYeD.
It Is MoSt StAnGe. OuR AvAtArS CoUlD noT fOcUs On CaPtAiN. ThEy CoUlD NoT CatCh It.

...It Has WaNdeReD OfF InTo ThE DeAD CiTy.
ThIs TrIaL Is A MoCkeRy WhEn ThE AcCcusEd CaN JuSt WaLK OfF.

ZeE CaPtAiN MuSt HaVe A WeAkNeSs, SoMeWheRe In ThE MeMoRy oF tHeSe ProPerTieS.

It WaS DiFfiCulT To TrAcK dOwN AlL InStaNcEs Of CaPtAin.

SoMeThiNg WaS InTeRfERiNg WiTh ThE ScAn.
SoMeThiNg ThaT DiD NoT BeLoNg iN ThE PaSt.
SoMeThiNg ThAt KePt MeDdLiNg WiTh EvErYtHiNg.

A WeApOn oF PoWer GrEaTeR ThaN tHe FuRnAcE Of ThE StArS.
An EmISsArY oF DeAtH, HiDdEn iN PlaIn SiGht.
BuT WhY? WhY DoEs DeAtH ItSeLf PrOtEcT tHe AcCuSeD?

We ToOk ToO LoNg. We AsSuMeD tHiS WoRlD wAs InAcTivE.
It WaS NoT.
The BiOmASs oF ThoUSaNdS CaNnoT NoT CoMpeTe WiTh A pLaNeTaRy DeFeNcE NeTwOrK iN PoSeSsiOn Of OrBiTaL IoN CaNnOns...


ThE UnIoN MuSt KnOw.
ThEy WiLl NoT Be PleAseD wItH OuR fAiLuRe.
wE TriEd. FoRgiVe Us.

...EnD TrAnSMiSsIOn.

Add a Comment:
No comments have been added yet.


Sat Feb 18, 2012, 1:58 AM

Zee Captain's note of summons had sealed my fate.

I felt a cold pull at my stomach. Something else was coming, re-awakened by the presence of Captain.
Something worse than the shuffling noises of the red beasts outside. The sound of falling cannonball.

Whatever that fleshy monstrosity was, it cared not for me, but it's presence here led ANNIE straight to me.

...And it came to pass on the morrow, that the goddess in the machine came upon me and she wanted vengeance.

She was online, the thermonuclear bombardment of the core databank cube had failed to shut her down.
She was still alive, after all these years.
The explosive crash-landing of the hunter machine through the old roof drew THEIR attention.
I was now between two wild beasts, out for my blood.

What was Protocol 1-1-3?
Memories of Directorate weapon codes lingered in my mind, poisoning my thought pattern.
Could it be?

...So this was to be my death.
ANNIE had let me know that she would spare no expense in crushing me like a bug, now that she knew exactly where I was.
How long would the signal take for activation?
I began to count down my last seconds.

Add a Comment:
No comments have been added yet.

We've just finished moving every message in every user's message center — all 6 billion of them — to a brand new set of database servers, and we managed to do it without anyone noticing. We didn't find much published material about the best way to do this, so we wanted to share a few details with you. It does get a little bit technical though so I won't be offended if you skip over some parts!

The data that powers message center was stored on one of our main sharded MySQL database clusters. That same cluster powers a lot of different features of the site and we recognised a lot of traffic was caused by this relatively simple but large set of data.

To free up capacity on that cluster, and help keep the site quick as we continue to grow, we decided to move the message center data to its own servers.

Some fun figures:
  • ~6 billion rows of data in 13 tables across 16 shards
  • Average 5k, daily peak of 10k queries per second for this data across the cluster
  • Average 100k, daily peak of over 150k rows read per second
  • Average 500, daily peak of over 3k rows written per second

So we had a pretty large set of data and the high traffic to it meant that we'd have to be very careful how we copied it over without losing any data, taking the message center down for a few days or otherwise affecting users.

To add a bit of fun to the project, the old tables were still using MyISAM storage engine as they've been around from before InnoDB's more recent rise to dominance. We decided it would be a good time to switch to InnoDB. If this doesn't mean much to you, then it means we decided to switch to a different type of database which has become a much better choice than it was a few years ago when we first designed message center.

How Much Hardware?

We started out not totally sure of how much hardware we would need because it was quite hard to get meaningful measurements of our current database server load when it was only a part of the traffic on the old cluster.

So to enable us to experiment and prove the new setup, we developed an extension for our PHP database API to allow us to duplicate query traffic from the current tables to the new cluster. We made it possible to control the percentage of read, write and delete queries mirrored individually though our distributed config system, and ensured that failures in the new cluster would not impact the real traffic.

After some experimentation we found a setup which worked well and gave us much more capacity to grow.

The new cluster has 8 servers as masters, replicating to 8 identical machines as a hot-backup.

Each machine has these specs:
  • 12 Cores
  • 96GB RAM
  • 200GB Intel SSDs

Since our existing cluster is sharded over 16 masters, we kept the data split into 16 logical shards and put 2 on each master machine in separate databases.

Syncing the data

Satisfied with the hardware setup, we needed to find a way to synchronise all the data. The initial import step was made more difficult by the move to InnoDB - it took several days to convert the most recent backup to InnoDB before the new cluster could start receiving new write traffic.

This left us with a gap in the data between the old and new clusters and we needed to find a way to get them back in sync without impacting the performance of the live cluster.

The nature of the data meant that even if we could efficiently copy only rows inserted within the missing period, that would still leave the data inconsistent where deletes happened during that time. People would start to see notifications they had already dismissed popping back up again.

So to be thorough, we needed to crawl all the data and ensure its consistency. Rather than copying everything again, we looked into ways to compare consistency between the live data and the new cluster without impacting users.


We couldn't find any documented technologies or techniques used by others that would work for us without taking the production cluster off-line so we came up with a solution that although not perfect, seems to have proven useful in this case.

It's all based on checksums.

A checksum is a number which is obtained by doing some calculations on the numbers that make up any piece of data on a computer. In general, if the checksum for two pieces of data is the same, they are very very likely to be identical. If the checksums are different, the two pieces of data are guaranteed to be different. The important thing is that using a checksum we can compare many thousands of rows with just one query to each cluster and very little data transferred.

We came up with a query that could calculate the CRC32 checksum of all the primary keys in a batch of in-boxes surprisingly efficiently. In this dataset, rows are never updated - just inserted or deleted - so primary keys are the only data points needed to ensure consistency.

The query looks something like this:

SELECT CRC32(SUM(row_cs)) as cs
  FROM (
      SELECT CRC32(CONCAT(folderid, type, itemid)) as row_cs
        FROM table
       WHERE folderid IN (1,2,3,4,...)
    ORDER BY folderid, type, itemid
  ) as tmp

[edit] rehael pointed out correctly in the comments below that the ORDER BY is unnecessary here. SUM is commutative so it doesn't actually matter if natural order is inconsistent in the inner query. This must have slipped in from an earlier attempt to accumulate rows without the sub query. In this case sort is on primary key so I guess it made little difference, but a good spot. I've left it in for posterity.[/edit]

[edit2] Another interesting potential flaw here is that CONCAT could produce false positives on some data sets. A more correct solution would be to use CONCAT_WS() with a delimiter char such that a row like (123, 456, 78) would not falsely checksum equal to (12, 345, 678). I'll credit pt-table-sync for this realisation (see bottom of article).[/edit2]

Using MySQL's EXPLAIN showed that this query didn't create a temporary table, and could select and sort efficiently using the primary key. This meant it is more likely to run quickly and not cause problems for real users of the site. In testing, we found it ran in under 200ms even for the biggest batches of folders we could find.

Despite the checksum being apparently quick, we still couldn't actually copy the out-of-sync data from big in-boxes from our master servers without locking extremely high-traffic tables, so we ran the checksums and copied the data from our hot backups.

To balance the amount of data copied with number of checksums needed, we first checksummed an entire batch of 50 in-boxes at a time. If there was a mis-match, we checksummed each in-box in the batch in turn and any that didn't match we deleted all rows from the new cluster and copied all rows from the old. There are more sophisticated options we could have used at this level to optimise for big in-boxes where not a lot changed, but none that were without downsides.

In reality the checksum query turned out not to be super-fast all the time, but we found more than 99.5% of the few million we ran in the end executed in under 200ms and over 99.8% in less than 1 second. In general checksum query load was minute compared to the load of the data-copying queries for big in-boxes.

You may not want to try this at home

There are a few flaws in this technique.

Firstly, the fact that we had to sync from replication slaves meant that any replication lag between master and slave caused inconsistency in the copied data. Once there were a few slow copy queries running, the replication lag would get worse. We found generally it kept up very well but we couldn't guarantee 100% consistency in one pass.

Secondly, even ignoring the replication lag, checksums could not be not atomic between the clusters - a user may have deleted or received a message in between the time you get the checksum from one cluster and the other.

So we knew a single pass through the data would not be sufficient. However, thanks to the checksum trick, we could continue running the sync job through the dataset and each time a few orders of magnitude less data would be found out of sync and need to be copied. Each pass also got substantially quicker as less data needed to be transferred.

After just 3 passes (several days for the first, a few hours for subsequent) we measured 99.9995% consistency. We called the job good and switched users over to read and write primarily from the new cluster. We're still writing to the old one for a week or so until we are totally sure of the new cluster.

Obviously this sort of slightly fuzzy consistency would not suitable for other data that may require absolute integrity (like customer's orders). For us, it's acceptable because:
  • a significant proportion of the < 0.0005% of in-boxes still measured as out of sync will actually be fine - just reported due to the measurement error noted above
  • statistically it is likely that any in-boxes genuinely out of sync are very large ones (100k+ notifications) and that their owners never look through and curate or remove them
  • in the highly unlikely case that a user did notice a discrepancy, it is more likely to be a notice they deleted re-appearing (they can fix that in one click)
  • these are not personal messages or unique content, just notifications of activity elsewhere on site

The useful take-aways

Despite not being a perfect solution to all live migration problems, I hope this has been somewhat interesting and may even help others with similar tasks. The exact plan we used is pretty specific to our situation but I think there are a couple of more general points to take away from this:
  • Migrating large amounts of data from one MySQL server cluster to another without affecting operation or losing writes is hard and apparently no one has come up with a really good solution yet
  • Under different circumstances, it may be possible to be more robust than this although probably at the expense of taking longer, requiring more resources, or making the service unavailable for some time
  • Checksum queries, although not a panacea, are a potentially useful tool that are not widely discussed for on-line MySQL consistency checking

Big thanks to chris and randomduck for their hard work on this upgrade.

[edit] A number of people pointed us at pt-table-sync from Percona's Toolkit. That script actually ends up doing something almost exactly like what we did here (with different checksumming techniques). I didn't know of it and somehow didn't find it when looking for similar solutions. It looks like a much more robust solution in general although would not (easily) have been able to take advantage of our distributed job cluster which enabled our migration to run sync queries on many folder chunks in parallel, so it probably would have taken much longer.[/edit]

Add a Comment:
No comments have been added yet.