$ cat dtc-api-extension.md
Hey o/
As this is a short project, that is now deprecated and doesn't work anymore, this article will be succinct.
When working on a chatbot, I started to implement API, to add more content to the bot. One of the websites was DTC. DTC (short for DansTonChat), is a website that compiles short funny IRC conversations, that are, due to the medium, themed around the 'geek' culture.
Unfortunately for my implementation, at the time, DTC didn't have any API. To remedy this, I decided to create a custom API, that will scrap the website when asked. I prefer to note here, that I asked the authorization of Remouk, the administrator of DTC, beforehand, to be sure that this will not penalize the website.
Usually, when working on an API, we have an OAuth authentication, to prevent spamming, and custom database request, to perform a quick, and optimized way of searching for the data the user is asking. Without access directly to DTC database, one way we can categorize and order the data, is by scraping the website.
To avoid that, as it can be really heavy in request, and I didn't want to overload the DTC website, we can also parse website pages for each request. In this case, we need to replicate the initial structure of the website. The API I decided to go with, was implementing the following structure :
|- quote
|- comment
|- random
|- users
|- comments
|- favoris
Each link would return something, for example, quote returns all information about a quote, with the id of the quote in the GET parameter.
When receiving a request about data, the website need to parse the page of DTC. To do so, we can use simple_html_dom.php by John Shclick. As we know how the website is organized, this will allow us to scrap the data of the page requested.
Then, as we have the data, parse into different variables. We can encode it in a json, to send it to the final user. We, of course, do not forget to change the header of the page we send, to be sure that it can be understood as a json file.