{"id":180,"date":"2012-10-25T17:41:01","date_gmt":"2012-10-25T05:41:01","guid":{"rendered":"http:\/\/site.stecman.ath.cx\/?p=180"},"modified":"2013-10-14T22:12:22","modified_gmt":"2013-10-14T10:12:22","slug":"javascript-for-everyday-scripting","status":"publish","type":"post","link":"\/blog\/2012\/10\/javascript-for-everyday-scripting","title":{"rendered":"JavaScript for everyday scripting"},"content":{"rendered":"<p>Though JavaScript tends to cop a lot of flak, I find it&#8217;s actually a pretty useful thing to know. No matter how much you may dislike JavaScript, the fact is that it&#8217;s part of almost every web browser since 1996, and it&#8217;s the only major, cross-browser client-side scripting option for the web (though <a href=\"http:\/\/www.dartlang.org\/\">Dart<\/a> is looking to join that rank). While I fully agree that writing large systems in JavaScript can be a pain, I can&#8217;t say the same thing about what&#8217;s in the name: scripting. I&#8217;m not talking about scripting in the sense of writing small programs, rather scripting the way you operate a command line; <strong>short, contextual snippets of code to get stuff done<\/strong>.<\/p>\n<p>Every now and then, I find myself popping open the JavaScript console in Google Chrome and writing a few lines of JavaScript to accelerate something I&#8217;m doing. While situations that warrant this are rare, there have been a few occasions where the JavaScript console has saved me huge amounts of menial clicking labour. It&#8217;s arguable that if you have to write code to do something on the web, then the creator(s) of the page you&#8217;re battling with didn&#8217;t do a good enough job. In reality however, most of the things I&#8217;ve accomplished with the JavaScript console have been far outside the scope of what the target pages were intended to do. So much to the point that only once have I used the JavaScript console to work around a real usability issue.<\/p>\n<p>Essentially, this all boils down to the fact that the data on every page is right there in your web browser, and in a highly accessible form at that. Why waste time doing things manually when you can automate?! It&#8217;s exactly the reason projects like <a href=\"https:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/greasemonkey\/\" title=\"Greasemonkey - Customize the way a web page displays or behaves, by using small bits of JavaScript. (for Mozilla Firefox)\">Greasemonkey<\/a> exist.<\/p>\n<p>Here&#8217;s a list for you to ponder:<\/p>\n<ul>\n<li><strong>Every component of a web page is accessible with JavaScript<\/strong> (ignoring a few cross-domain nuances with iframes).<\/li>\n<li>You don&#8217;t just get access to the raw data of a page; you get a highly organised hierarchy (the DOM).<\/li>\n<li>The page is on your computer, so you can do whatever you like to it. No-one can stop you (except maybe with some preemptively written defensive JavaScript, but that&#8217;s a bit silly).<\/li>\n<\/ul>\n<p>As a crude example, lets say you wanted to open every link on a page in a new window\/tab. You could achieve that like this:<\/p>\n<pre class=\"prettyprint \">\r\n\/\/ Get all the anchor (link) elements on the page\r\nvar links = document.getElementsByTagName('a');\r\n\r\n\/\/ Loop through the array of links, opening each one\r\nfor( var i=0, c=links.length; i&amp;lt;c; i++){\r\n    window.open(links[i].href);\r\n}\r\n<\/pre>\n<p><strong>Note that any half-decent pop-up blocker will stop this working<\/strong> (you easily can disable your pop-up blocker if you really want to try it out).<\/p>\n<h2>Mining Bebo with JavaScript<\/h2>\n<p>In one of my furious efforts to digitally archive everything relevant in arms reach, I decided it would be a good idea to take some old conversations I had on Bebo. To save myself the effort of manually retrieving pages, I wrote an email to Bebo&#8217;s support asking for a dump of all comments to and from my account, and a few hours later I received this rather helpful response:<\/p>\n<blockquote>\n<p>Dear Bebo User:<\/p>\n<p>Thank you for your feedback. We always enjoy hearing from our users.<\/p>\n<p>Jordan<\/p>\n<\/blockquote>\n<p>I definitely wouldn&#8217;t have done what I did next for the likes of Facebook or twitter (not that it would be necessary with their public APIs), however <a href=\"http:\/\/en.wikipedia.org\/wiki\/Bebo\" title=\"The Wikipedia entry for Bebo outlines this pretty well\">Bebo wasn&#8217;t exactly rocketing through the inter-sky<\/a> and I didn&#8217;t want to lose two years of conversation with my partner because someone pulled the plug on a failing social network. I wrote back kindly pointing out that Jordan hadn&#8217;t in-fact attempted to answer my question at all, and soon received this curt reply:<\/p>\n<blockquote>\n<p>That is not a service that we offer.<\/p>\n<\/blockquote>\n<p>With the easy option off the table, I set out to harvest my comment wall from Bebo, and I did. Very successfully. With less than 100 lines of JavaScript, I turned pages and pages of old Bebo comments into 697 rows in a MySQL table on my server in about 5 minutes flat.<\/p>\n<h3>The harvesting code<\/h3>\n<p>I used an <code>iframe<\/code> as a viewport and ran snippets of JavaScript in the window to retain state above page changes. Once a small loop had loaded every page and scraped their comments, I packaged everything up in an <code>XMLHttpRequest<\/code> POST request and fired it off to a script on my local server for database insertion. The key component in this operation was the iframe:<\/p>\n<pre class=\"prettyprint \">document.body.innerHTML = '&lt;iframe width=&quot;100%&quot; height=&quot;1000px&quot; src=&quot;'+window.location+'&quot;&gt;&lt;\/iframe&gt;';<\/pre>\n<p>The browser allowed me to access the contents of this iframe with JavaScript <em>because I created it inside a page on Bebo&#8217;s own domain<\/em>, and it&#8217;s content was also from Bebo&#8217;s domain. This line of code essentially moves the current page into an iframe on it&#8217;s own page, retaining the context of the original page as far as the browser is concerned (Inception-frame?). After this trick, the rest of the code I ran through the console was fairly straight forward:<\/p>\n<pre class=\"prettyprint \">\r\n\/\/ Get all comment elements\r\nvar comment_elements = window.frames[0].document.getElementById('comment-list').getElementsByTagName('LI');\r\n\r\nfor (var i=0; i&amp;lt;comment_elements.length; i+=10) {\r\n    var el = comment_elements[i],\r\n        comment = {};\r\n\r\n    \/\/ Ignore a specific type of comment\r\n    try {\r\n        comment.from = el.childNodes[3].innerHTML;\r\n        if(comment.from.search(\/^&amp;lt;\/) != -1)\r\n            comment.from = el.childNodes[5].innerHTML;\r\n    } catch(e) {\r\n        console.log(&quot;Ignored Moble&quot;);\r\n    }\r\n\r\n    comment.date = el.getElementsByClassName('comment-timestamp')[0].innerHTML;\r\n    comment.text = el.getElementsByClassName('comment-text')[0].innerHTML;\r\n\r\n    if(comment.from == &quot;Steph&quot; || comment.from == &quot;Stephen&quot;){\r\n        comments.push(comment);\r\n    }\r\n}\r\n<\/pre>\n<p>Once I had the data I wanted, sending cross-domain requests was a breeze. While I could have added a few headers to my server&#8217;s response to stop the browser complaining about and &#8216;blocking&#8217; cross-domain requests, <strong>I didn&#8217;t need to do anything more than send the requests<\/strong>. The browser complained away, however because it needed to send my requests to read the cross-domain permission headers in the first place, my server happily digested the incoming data and I didn&#8217;t care about the discarded response.<\/p>\n<p>While I could have gathered this data using any old language by downloading the HTML pages, the requirement for a logged-in session to view the pages I wanted was a whole other can of worms. In the end, running JavaScript in the context of an authenticated session in a browser made life a lot easier.<\/p>\n<p><em>Update, August 2013: My concerns about losing this data were well founded: the old Bebo is now inaccessible. Apparently some data will be available in the future after Bebo gets refurbished, but I&#8217;m still glad I got this stuff off when I did.<\/em><\/p>\n<h2>And with that&#8230;<\/h2>\n<p>I don&#8217;t think that everyone should have JavaScript under their belt, but it has certainly been invaluable to me. If you&#8217;re interested in learning JavaScript, the <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/JavaScript\" title=\"MDN: JavaScript\">Mozilla Developer Network<\/a> is an excellent resource, and you could always pop open the JavaScript console in your browser.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Though JavaScript tends to cop a lot of flak, I find it&#8217;s actually a pretty useful thing to know. No matter how much you may dislike JavaScript, the fact is that it&#8217;s part of almost every web browser since 1996, and it&#8217;s the only major, cross-browser client-side scripting option for the web (though Dart is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[33],"tags":[34,56,35],"_links":{"self":[{"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/posts\/180"}],"collection":[{"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/comments?post=180"}],"version-history":[{"count":51,"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/posts\/180\/revisions"}],"predecessor-version":[{"id":823,"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/posts\/180\/revisions\/823"}],"wp:attachment":[{"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/media?parent=180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/categories?post=180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stecman.co.nz\/wp-json\/wp\/v2\/tags?post=180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}