Shop Forum More Submit  Join Login
About Varied / Hobbyist Official Beta Tester David RefouaMale/United States Groups :icondeviousbots: DeviousBots
Bots, Bots everywhere!
Recent Activity
Deviant for 5 Years
4 Month Core Membership
Statistics 17 Deviations 2,402 Comments 29,500 Pageviews

Donate

DRSDavidSoft has started a donation pool!
12,815 / 16,000
Here's a donation box if you would like to donate some points.

You must be logged in to donate.
:icondailygiveget::icondailygiveget::icondailygiveget::icondailygiveget::icondahub::icondailygiveget::icondailygiveget::icondahub::icondahub::icondailygiveget:

If DeviantArt updates its theme/skin, would you prefer a dark one or a light one? 

51%
153 deviants said Dark + New Green
24%
72 deviants said I really prefer this current style
21%
62 deviants said Light + New Green
4%
11 deviants said None/Other (Please comment)

How to create your own bot from scratch

Journal Entry: Tue Apr 10, 2018, 4:28 AM


Greetings! 
– v1.2 (added some more info)

Someone inspired by dADroid-bot recently asked me to help them in the right direction of creating their own bot, and I wanted to write a complete guide on how a bot is made. Since this seemed to be something that other people might want to read in the future, I decided to write and share a comprehensive guide, listing all the required tools and skills to make a new bot from scratch without any past experience.

Now, I should mention that there are in fact third-party tools and services that can be used to create and/or replace any part of this guide, but considering that those services do or use more or less the same things explained here, I'd recommend to be at least familiar with the concepts explained here, and the basics of what is happening behind the scenes to run a bot.

I'd like to point out a few essential tools & skillsets required to start a new project like this, and I hope you'll make an amazing bot here!

With that said, let's get started!

Introduction – What is a bot?

a robot is simply a piece of software designed to work continuously behind the scenes to do what real humans do, automatically. Some bots are unfortunately made with malicious intent, but most bots are created to help people easily & quickly do what they'd otherwise regularly do by hand.

On DeviantArt, this generally means reading & writing comments, posts, journals and interacting with people.
(For more information about Good vs. Bad bots, have a look at here.)

There's also another category of completely different Deviant Art bots which happen to interact with the Deviant Art Messaging Network, which BotLab has had covered very well. Those bots are specific to chat system this site runs. This category is out of the scope of this tutorial.

Step 1 – Programming language

The first step to create a bot (or a software of any kind) is to be able to code.
Do you have any past experience with coding? If so, you might be able to focus writing your bot with that. If not, no worries, you'll need to start learning a new Programming Language. Which programming language to choose is not that important as long as it's capable of connecting to DeviantArt.

You'll just need to understand the basics of computer logic and should probably be comfortable to use the language on a regular basis. As an example for the programming language, dADroid-bot is coded in PHP, a standard web-development language. Other bots, such as dAbot (created by my friend Kishan-Bagaria) use Python, and some like dAhub I believe use Java (not to be confused with JavaScript).

I'll try to give a short description for each language, and I hope you find the right amd the most suitable one to use for you.

(Remember: memorizing it is not required, instead you should learn the logic behind it.)

  • Ruby: a beautiful, simple to understand and & easy to use but powerful tool to start programming.
    Try it here: tryruby.org

  • Python: a very popular scripting language, has a great, powerful and moderately easy to master syntax.
    Try it at: learnpython.org

  • Lua: a very flexible and easy scripting language. Here's an excellent YouTube video: www.youtube.com/watch?v=S4eNl1…

Once you've familiarized yourself with the concept, you can try some more advanced/complex languages in the future:

  • JavaScript (not to be confused with Java): the defacto standard scripting language. it's powerful yet easy, and originated from designing web pages and currently focused for modern web-apps.
    Try it at: www.javascript.com/try
    (I highly recommend learning this in the long run!)

  • PHP: the most-used web programming language, which only runs in the back-end (like servers, though it's possible to run on Desktop computers and your laptop too). It's powerful but some consider it to be ancient, and most modern developers try to avoid it. A very dangerous tool if handled uncarefully. A powerful, sharp tool if handled with care. :)
    dADroid is written mostly in PHP.

  • Java (not JavaScript): a powerful, moderately easy to understand and object-oriented language, which is somehow similar to JavaScript, yet completely different. Developed by Oracle Corporation and it's in wide use by many businesses. You don't need to bother yourself with it, at least for now. I believe dAhub was written in Java.

  • C#: developed by Microsoft, originally based on C and C++ syntax. Much better than ancient and unmaintained counterpart VB (visual basic) language, which was also designed by Microsoft. Until recently, it could only run on Windows operating systems (which meant not being able to run on Linux servers), though a minimalistic version for Linux has been recently released.
    It has a somehow similar syntax to JScript (Microsoft's fork of JavaScript.)
    Used widely in business environments, not exactly the most suitable for DIY projects like this, but has great potential, and as such is widespread, and hence has great support on websites like StackOverflow.com.
    Again, you do not bother yourself with it, just now. Might be a handy tool if you've picked up programming languages in academic classes (like college and university.)

Step 2 – API (connecting to DeviantArt)

Now that you've worked with some languages and picked up a general syntax (and hopefully chosen one of them to use), it's time to write an interface for your bot.

Using an API (application programming interface) your bot can talk to DeviantArt servers, and vice versa. If you imagine two pieces of code (one is your bot and the other one is DeviantArt server) are commuting with each other, the method they send and receive data is called an API. Kinda like what texting is for humans, if you will. 

Wikipedia has a great (and in-depth) article on APIs, if you want to read more. (Not bad reading material!)

DeviantArt uses two distinct set of APIs:
  1. a public one which DeviantArt provides here:
    www.deviantart.com/developers/

  2. an internal one called DiFi (DeviantArt Interactive Fragmented Interface) which they use themselves for calls between their site and their server.

    some kind souls have documented DiFi here:
    github.com/danopia/deviantart-…
a bot can use either (or even both) of these APIs to interface their bot with DeviantArt.

The public APIs are somehow limited, and you're required to register a key for your application to work with. However, it's free, documented regularly, and super-easy to use, and less prone to suddenly breaking.

The DiFi API is much more powerful, doesn't require signing up for a key, and is more popular with the DeviantArt modding community. However, DeviantArt site owners meant it to be only used for their own internal stuff, so it's a bit less reliable, can change at any given time without prior warning, and thus could potentially break your program. However, since DeviantArt hasn't really changed for several years, it's rare and unlikely that it completely remove this API.
(That's not to say instances of a bot completely breaking down in the past haven't occurred at all (like dAhub), but usually, developers are able to fix them and bring them back online completely, usually at a short time.)

Another concept which you should be familiar with is HTML scraping, which is the practice of fetching and loading the HTML code that is meant for the browsers from the DeviantArt servers into the bot and parsing it.

Professionals most often recommend using the official and public APIs, and you don't hear one recommending you to use the internal APIs. Against their warnings, however, dADroid, dAbot, (and dAhub to my knowledge) all use the DiFi internal API with the addition of some HTML scraping, because it's more powerful and portable.

Basically, anything you can do with the website can be done using DiFi, but they might try to block you (or your bot) – especially if your bot is made with malicious intent.

dADroid has a great HTML scraping and DiFi parsing utility built right into it.

The unofficial route gives you much more power but is harder to maintain. You may, however, choose to use the official way, which is a bit less powerful but easier to work with.

So what to use?

If you choose public APIs, everything is documented, but you need to register a key, and fewer functions are available to you.

If you prefer to have more functions available to use at the cost of unofficial support, you can go the DiFi way. If so, you can either write the code for DiFi yourself, use the already made samples, or politely ask someone to write it for you.
You can use these as samples to write your own. The one for dAhub is closed-source, meaning that you can not obtain the source code from him.

dADroid-bot (the one that I've created) is open-source, and actively maintained, but at the time of creating it I didn't have any intentions of releasing its source code to the public, so it's a bit messy.

Nonetheless, if you'd like to use PHP, you can use dADroid's source in your project. You can see a list of version history here: dadroid.ir/Changelog.txt


Step 3 – means of storage

So, at this point you've created a bot which can connect to DeviantArt, read its content, and post some results onto it.

Now, as the APIs are eyes and ears of your bot, you need a brain to process and store the content your bot receives.

What use would a bot have if it forgets everything whenever you close it?

You can use two primary methods of storing the content of your bot for later retrieval and use: a) as files on your disk and b) using a Database software.

For file-based storage, have a look at XML, YAML , and JSON. Don't get scared by lines of text! They're merely written to help your bot remember what's going on after each startup.

However, modern and complex bots can't rely on text files to accurately and efficiently store their data. So you need to use another tool to handle your data. 

Nowadays, the SQL (read SEQUEL) which stand for Structured Query Language is the standard way of writing and accessing data in Databases.

The most popular SQL servers (which you can use in your project) are MySQL and MariaDB at the moment. They are sister databases and you can use them interchangeably. The first one is currently developed by Oracle, the same company who maintains Java (not to be confused by JavaScript).

If you've ever heard of WordPress, it uses one of these Databases to hold its data, and I'm pretty sure that DeviantArt also uses something similar.

There's a lighter version of SQL which doesn't need a server and resides on disk, called SQLite. You can use it for your starting project. Fun Fact: Google uses SQLite in their Android OS all the time. So if you currently have a phone with Android as its OS, there's a pretty good chance that your phone uses SQLite!

Microsoft also uses their own propriety and non-free fork of SQL server, which is focused on businesses and you don't have to worry about.

All of these SQL variants (with the exception of Microsoft's) are free and open-source, you can download the software right away and use them in your projects.

Here are some great resources to learn SQL:
  1. www.w3schools.com/sql/
  2. www.codecademy.com/learn/learn…

Step 4 – logic, user settings & GUI

Now that your bot can connect to DeviantArt, get data, write to it, and remember what to do; you need to write the logic for what it actually needs to do.

The best way to do this is to write down your requirements for a bot, organize it into smaller block diagrams, and write the code for each block of your diagram. It's easier to follow (and debug) your bot's logic that way as it gets bigger, especially if you comment your work.

You also need to devise a means of controlling and "configuring" your bot. You can write an external web-page for its settings, or use DeviantArt comments. It's up to you! Once the command is received, the result will be saved in the Database.

At one point, you'll also probably need to implement timers in your logic, which on Linux servers are achieved by using cron jobs. Best idea to familiarize yourself with the concept, but no need to worry about it, for now.

When coding, it's best to always follow DRY practice: Don't Repeat Yourself. This means avoid copy-pasting code, which trust me, makes everything a hundred times harder. Instead, split your code into different meaningful and re-usable parts.

You can read more about on how to follow this pattern in a carefully designed fashion called MVC:
  1. Understanding Model-view-controller
  2. What is MVC?
  3. Wikipedia's on Model–view–controller
  4. MVC Pattern Tutorial
I'm not going to get deep into this, as there are already a lot of good articles written. Just remember, keep everything organized, and keep your code clean and well-commented, even if you're the only one working on the project, you wouldn't know if someone else would work on it tomorrow. Otherwise, you'll get into some unpredicted issues soon, like I have with the dADroid's code getting so messy that I need a full rewrite. :)

By the way, if you're going to version your bot (and you should), it's best to follow the Semantic Versioning practices. Remember: You probably won't be the only one working on the code, forever.


Step 5 – running the bot on a server

The last step is to run your bot somewhere which can it keep running in the background, as long as you wish.

For this task, generally, a server is used to keep your bot running. You can host your own server (which definitely isn't recommended, but I personally do anyways), or you can go the easier route, and buy a hosting plan (renting a server dedicated to your project.)

Generally, VPS hosting services are really cheap, and can run your code in the cloud. I recommend using them. Or, you can also choose to run your bot from your own personal computer, which means you need to keep your PC always on to keep your bot continuously running. If your computer (or a server for that matter) is turned off, it equals your bot not running.

You should choose your server plan very carefully according to your needs, especially if you intend to keep the bot running for a long time.

If you develop your bot in PHP (like I did) you can buy even cheaper plans meant for website hosting services, which could also host your bot program.

Final Notes – social side of the bot

Please try to keep your bot polite, and avoid spamming people. That's a very bad thing to do!

Don't do things that DeviantArt staff wouldn't like, they'd block you for no sensible reason, faster than what you'd think. I've learned this the hard way.

And it has become a standard convention on DeviantArt to mark your bot's generated messages with "I'm a bot" or similar notes. It's especially important if your bot generates replies to somehow identify it as a bot. (Like any instance of both dAhub and dADroid-bot. All of the popular bots on DeviantArt do this, to my knowledge.)

Conclusion

And there you go, a list of all essential points I could think of for a working and practical Deviant Art bot! I hope you learn new things along the way and have a great experience writing your own bot.

Be sure to check DeviousBots for some examples of useful bots around DeviantArt, and submit your own bot there if you made a great one!

You are encouraged to comment any questions you have or any problems that might occur in the future. I'll try my best to answer them when I can!

I tried to list everything that came to my mind, but I might update this journal in the future if something else also needs to be mentioned.

Have a great day.


Appreciation Acknowledement

Hi there!
• I'd like to show my gratitude towards everyone who takes time to post a comment here!
• Also if you traded a llama with me in a, thank you as well!
• Since I obviously can't reply to all comments, please accept my "you're quite welcome" if you've posted a "thank you" message!

Comments


Add a Comment:
 
:iconrevan005:
Revan005 Featured By Owner Aug 23, 2018
Thanks you for the llama!
Reply
:iconagnosticdragon:
AgnosticDragon Featured By Owner Jul 13, 2018  Hobbyist Writer
Happy birthday!
Reply
(1 Reply)
:iconcamaro--princess:
Camaro--Princess Featured By Owner Jul 13, 2018  Hobbyist General Artist
Spideypool - Happy Birthday 1Spideypool - Happy Birthday 2 
Reply
(1 Reply)
:iconjjjmadness:
JJJMadness Featured By Owner Jul 13, 2018  Hobbyist Digital Artist
 candlescandlescandlescandlescandlescandlescandlescandles
 Scalpel (Left)Tini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeTini Vintage Syringe
       bunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley faces Eyeball  bunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley faces
       bunch of smiley facesbunch of smiley facesTini Cleaver (Right)  Tini Preserved Heart Moving Eye Ball Cup Cake 50x50 icon  Broken Heart  Moving Eye Ball Cup Cake 50x50 icon  Tini Preserved Heart Tini Cleaver (Left)bunch of smiley facesbunch of smiley faces
       bunch of smiley facesPill AvatarMoving Eyeballs Cake Type 2 50x50 icon Virtual Retro LED - HVirtual Retro LED - AVirtual Retro LED - PVirtual Retro LED - PVirtual Retro LED - Y Moving Eyeballs Cake Type 2 50x50 iconPopsiclebunch of smiley faces
  GHOST HEART Moving Eyeballs Cake 50x50 iconVirtual Retro LED - BVirtual Retro LED - IVirtual Retro LED - RVirtual Retro LED - TVirtual Retro LED - HVirtual Retro LED - DVirtual Retro LED - AVirtual Retro LED - YMoving Eyeballs Cake 50x50 icon GHOST HEART 
Tini Vintage Syringe leftTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeEyesTini EyeScalpel (Right)
 candlescandlescandlescandlescandlescandlescandlescandles
Reply
(1 Reply)
:iconiamthejam:
iamthejam Featured By Owner Jul 13, 2018
¡¡¡Feliz cumpleaños!!!
Reply
(1 Reply)
:iconsleestakgod:
sleestakgod Featured By Owner May 28, 2018
Thanks for the Llama.
Reply
:icontatihienahtf:
tatihienahtf Featured By Owner May 15, 2018  Hobbyist General Artist
Thanks for the llama!!Hooray Llama Badges 
Reply
(1 Reply)
:iconcarsfan16:
carsfan16 Featured By Owner May 14, 2018  Student Artist
thank you for the llama dude
Reply
(1 Reply)
:iconcalucifer13:
Calucifer13 Featured By Owner May 14, 2018  Professional Traditional Artist
Thanks for the llama!
Reply
(1 Reply)
:iconloki-icon:
Loki-Icon Featured By Owner Apr 24, 2018  Hobbyist Digital Artist
Have already given you a llama i do not know when Thanks for the llama back :)
Reply
:iconthewalkingbaka:
TheWalkingBaka Featured By Owner Apr 11, 2018  Hobbyist Digital Artist
thanks for my 50th llama now im an albino!
Reply
(1 Reply)
:iconcocoaandtea:
CocoaAndTea Featured By Owner Apr 10, 2018  Hobbyist Digital Artist
oof I accidentally unwatched you ;-; 
Hope you are doing fine this year.
Reply
(1 Reply)
:iconlink2time:
Link2Time Featured By Owner Dec 2, 2017  Professional Traditional Artist
Do you have the old code for daDroid anywhere? I'm aware many features were removed in the last year and am interested in reprogramming it myself.
Reply
(2 Replies)
:iconwolf5winer:
wolf5winer Featured By Owner Oct 17, 2017  Hobbyist Artist
Thanks for the llama. ^^
Reply
:iconbigclaudia:
BigClaudia Featured By Owner Oct 7, 2017  Hobbyist General Artist
are you creator of this?
getllamas.com
Reply
(1 Reply)
:iconbeautyandnotthebeast:
beautyandnotthebeast Featured By Owner Sep 17, 2017  Hobbyist General Artist
Thanks for the llama.
Reply
:iconnonaesthetic:
nonaesthetic Featured By Owner Aug 5, 2017
hey my dude, dahub is broken.. sorry, a lot of people have probs told you this already. I just want to make sure you know!
Reply
(1 Reply)
:iconrubyrebecca123:
rubyrebecca123 Featured By Owner Jul 13, 2017  Hobbyist Digital Artist
Happy birthday mang
Reply
:iconmirycchi:
Mirycchi Featured By Owner Jul 13, 2017
     :iconhappybirthday2plz::iconblue-hplz::iconblue-aplz::iconblue-pplz::iconblue-pplz::iconblue-yplz::iconhappybirthday2plz:
:iconblue-bplz::iconblue-iplz::iconblue-rplz::iconblue-tplz::iconblue-hplz::iconblue-dplz::iconblue-aplz::iconblue-yplz:
Reply
:iconlost-swords:
Lost-Swords Featured By Owner Jul 13, 2017  Hobbyist Digital Artist
HAPPY BIRTHDAY, PartyPartyPartyParty
Reply
:iconjjjmadness:
JJJMadness Featured By Owner Jul 13, 2017  Hobbyist Digital Artist
  Spin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it good
Pixel Icon: Ink-Heart (F2U!)Sweetskull GlowSweetskull: Purple and TealSweetskull: Purple and IndigoSweetskull: Magenta and BlueSweetskull: Salmon and PurpleSweetskull: Orange and PinkSweetskull: Indigo and TealSweetskull GlowSweetskull: Purple and TealSweetskull: Purple and IndigoSweetskull: Magenta and BlueSweetskull: Salmon and PurpleSweetskull: Orange and PinkSweetskull: Indigo and TealSweetskull GlowSweetskull: Purple and TealSweetskull: Purple and IndigoSweetskull: Magenta and BlueSweetskull: Salmon and PurpleSweetskull: Orange and PinkSweetskull: Indigo and TealSweetskull GlowSweetskull: Purple and TealSweetskull: Purple and IndigoSweetskull: Magenta and BlueSweetskull: Salmon and PurpleSweetskull: Orange and PinkSweetskull: Indigo and TealSweetskull GlowSweetskull: Purple and TealSweetskull: Purple and IndigoSweetskull: Magenta and BlueSweetskull: Salmon and PurplePixel Icon: Ink-Heart (F2U!)
     bunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley faces:souleatermoon:bunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley facesbunch of smiley faces
     bunch of smiley facesbunch of smiley faces
Red Eye:oldtv:Moving Eyeballs Cake 50x50 iconBloody Rain Good NightMoving Eyeballs Cake 50x50 icon:oldtv:Red Eyebunch of smiley facesbunch of smiley faces
     bunch of smiley facesPastel Purple Eye
Flowey Omega (avatar size) - UNDERTALE - Icon GIFVirtual Retro LED - HVirtual Retro LED - AVirtual Retro LED - PVirtual Retro LED - PVirtual Retro LED - YFlowey Omega (avatar size) - UNDERTALE - Icon GIFPastel Purple Eyebunch of smiley faces
    
Blue EyeTVVirtual Retro LED - BVirtual Retro LED - IVirtual Retro LED - RVirtual Retro LED - TVirtual Retro LED - HVirtual Retro LED - DVirtual Retro LED - AVirtual Retro LED - YTVBlue Eye
Pixel Icon: Ink-Heart (F2U!)Sweetskull: Salmon and PurpleSweetskull: Magenta and BlueSweetskull: Purple and IndigoSweetskull: Purple and TealSweetskull GlowSweetskull: Indigo and TealSweetskull: Orange and PinkSweetskull: Salmon and PurpleSweetskull: Magenta and BlueSweetskull: Purple and IndigoSweetskull: Purple and TealSweetskull GlowSweetskull: Indigo and TealSweetskull: Orange and PinkSweetskull: Salmon and PurpleSweetskull: Magenta and BlueSweetskull: Purple and IndigoSweetskull: Purple and TealSweetskull GlowSweetskull: Indigo and TealSweetskull: Orange and PinkSweetskull: Salmon and PurpleSweetskull: Magenta and BlueSweetskull: Purple and IndigoSweetskull: Purple and TealSweetskull GlowSweetskull: Indigo and TealSweetskull: Orange and PinkSweetskull: Salmon and PurpleSweetskull: Magenta and BlueSweetskull: Purple and IndigoSweetskull: Purple and TealSweetskull GlowPixel Icon: Ink-Heart (F2U!)
  Spin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it goodSpin it good
                                                       Rainbow Letter: S (Static) Rainbow Letter: E (Static) Rainbow Letter: L (Static) Rainbow Letter E (Animated) Rainbow Letter: C (animated) Rainbow Letter: T (Static)    Rainbow Letter: A (Static)    Rainbow Letter: G (Static) Rainbow Letter: I (Static) Rainbow Letter: F (Static) Rainbow Letter: T (Static)

First Gift              Second Gift                 Third Gift                  Fourth Gift               Fifth Gift
Sixth Gift                                              
Seventh Gift                                              Eighth Gift
Reply
:iconjimmyneutronfreak1:
jimmyneutronfreak1 Featured By Owner Jul 13, 2017  Student General Artist
HAPPY BIRTHDAY, COLLAB BUDDY!!!  I know you're eight and a half hours ahead of me, so I hope you had an awesome day!! :party: Can't wait to see what you come up with next!! :eager: :happybounce:
Reply
:iconiamthejam:
iamthejam Featured By Owner Jul 13, 2017
¡¡¡Feliz cumpleaños!!!
Reply
:iconjokerfake:
jokerfake Featured By Owner Jul 2, 2017  Hobbyist General Artist
Thanks for the llama
Reply
Add a Comment: