Data Mining With Node.js

Learn how to use Node.js in advance as data mining tools

REPL Driven Development

January 27, 2014

When I describe my current workflow I use the TLA RDD, which is short for REPL Driven Development. I've been using REPL Driven Development for all of my production work for awhile now, and I find it to be the most effective workflow I've ever used. RDD differs greatly from any workflow I've used in the past, and (despite my belief that it's superior) I've often had trouble concisely describing what makes the workflow so productive. This entry is an attempt to describe what I consider RDD to be, and to demonstrate why I find it the most effective way to work.

RDD Cycle

First, I'd like to address the TLA RDD. I use the term RDD because I'm relying on the REPL to drive my development. More specifically, when I'm developing, I create an s-expression that I believe will solve my problem at hand. Once I'm satisfied with my s-expression, I send that s-expression to the REPL for immediate evaluation. The result of sending an s-expression can either be a value that I manually inspect, or it can be a change to a running application. Either way, I'll look at the result, determine if the problem is solved, and repeat the process of crafting an s-expression, sending it to the REPL, and evaluating the result.

If that isn't clear, hopefully the video below demonstrates what I'm talking about.

If you're unfamiliar with RDD, the previous video might leave you wondering: What's so impressive about RDD? To answer that question, I think it's worth making explicit what the video is: an example of a running application that needs to change, a change taking place, and verification that the application runs as desired. The video demonstrates change and verification; what makes RDD so effective to me is what's missing: (a) restarting the application, (b) running something other than the application to verify behavior, and (c) moving out of the source to execute arbitrary code. Eliminating those 3 steps allows me to focus on what's important, writing and running code that will be executed in production.

Feedback

I've found that, while writing software, getting feedback is the single largest time thief. Specifically, there are two types of feedback that I want to get as quickly as possible: (1) Is my application doing what I believe it is? (2) What does this arbitrary code return when executed? I believe the above video demonstrates how RDD can significantly reduce the time needed to answer both of those questions.

In my career I've spent significant time writing applications in C#, Ruby, & Java. While working in C# and Java, if I wanted to make and verify (in the application) any non-trivial change to an application, I would need to stop the application, rebuild/recompile, & restart the application. I found the slowness of this feedback loop to be unacceptable, and wholeheartedly embraced tools such as NUnit and JUnit.

I've never been as enamored with TDD as some of my peers; regardless, I absolutely endorsed it. The Design aspect of TDD was never that enticing to me, but tests did allow me to get feedback at a significantly superior pace. Tests also provide another benefit while working with C# & Java: They're the poorest man's REPL. Need to execute some arbitrary code? Write a test, that you know you're going to immediately delete, and execute away. Of course, tests have other pros and cons. At this moment I'm limiting my discussion around tests to the context of rapid feedback, but I'll address TDD & RDD later in this entry.

Ruby provided a more effective workflow (technically, Rails provided a more effective workflow). Rails applications I worked on were similar to my RDD experience: I was able to make changes to a running application, refresh a webpage and see the result of the new behavior. Ruby also provided a REPL, but I always ran the REPL external to my editor (I knew of no other option). This workflow was the closest, in terms of efficiency, that I've ever felt to what I have with RDD; however, there are some minor differences that do add up to an inferior experience: (a) having to switch out of a source file to execute arbitrary code is an unnecessary nuisance and (b) refreshing a webpage destroys any client side state that you've built up. I have no idea if Ruby now has editor & repl integration, if it does, then it's likely on par with the experience I have now.

Semantics

It's important to distinguish between two meanings of "REPL" - one is a window that you type forms into for immediate evaluation; the other is the process that sits behind it and which you can interact with from not only REPL windows but also from editor windows, debugger windows, the program's user interface, etc.
It's important to distinguish between REPL-based development and REPL-driven development:
REPL-based development doesn't impose an order on what you do. It can be used with TDD or without TDD. It can be used with top-down, bottom-up, outside-in and inside-out approaches, and mixtures of them.
REPL-driven development seems to be about "noodling in the REPL window" and later moving things across to editor buffers (and so source files) as and when you are happy with things. I think it's fair to say that this is REPL-based development using a series of mini-spikes. I think people are using this with a bottom-up approach, but I suspect it can be used with other approaches too.
-- Simon Katz

I like Simon's description, but I don't believe that we need to break things down to two different TLAs. Quite simply, (sadly) I don't think enough people are developing in this way, and the additional specification causes a bit of confusion among people who aren't familiar with RDD. However, Simon's description is so spot on I felt the need to describe why I'm choosing to ignore his classifications.

RDD & TDD

RDD and TDD are not in direct conflict with each other. As Simon notes above, you can do TDD backed by a REPL. Many popular testing frameworks have editor specific libraries that provide immediate feedback through REPL interaction.

When working on a feature, the short term goal is to have it working in the application as fast as possible. Arbitrary execution, live changes, and only writing what you need are 3 things that can help you complete that short term goal as fast as possible. The video above is the best example I have of how you go from a feature request to software that does what you want in the smallest amount of time. In the video, I only leave the buffer to verify that the application works as intended. If the short term goal was the only goal, RDD without writing tests would likely be the solution. However, we all know that that are many other goals in software. Good design is obviously important. If you think tests give you better design, then you should probably mix both TDD & RDD. Preventing regression is also important, and that can be accomplished by writing tests after you have a working feature that you're satisfied with. Regression tests are great for giving confidence that a feature works as intended and will continue to in the future.

REPL Driven Development doesn't need to replace your current workflow, it can also be used to extend your existing TDD workflow.

[…]

5 WAYS TO LEARN ABOUT COMPETITORS ADVERTISING STRATEGIES

September 29, 2014

When it comes to online advertising, it is important to have a solid and scalable marketing strategy.
At the same time, is one of the main features of the SEM, that the efforts of competition affect the performance of your online campaigns more than with traditional marketing methods.
How to beat your competitors, you must first know what their strategies are. Ideally, you also connect the dots between ads, offers, landing pages and social media accounts.

There are several ways to learn more about your competitors' advertising campaigns. Here are five that I have used that are useful and effective.

AdWords auction Insights

The auction Insights report is in an AdWords campaign or ad group level are retrieved, and gives you valuable information about what other companies are bidding on your keywords in your geographic target markets. In addition, she shows her impressions share, compared to your own, as well as how often (in percent) that showed you and your competitors at the top of the page above the fold significance.
While these findings may be helpful, they can easily be misinterpreted. For example, the impression share of a large global company and a small local business could be significant. So could be the reasons for advertising. Not all companies are for uncomplicated conversions looking to AdWords, or have large budgets for brand campaigns on the Google search network, the lead share in a great impression.

AdWords Ad Preview Tool

The Google AdWords Ad Preview Tool is underestimated. The tool allows you to compare and display of specific competitors such as displaying various competitors with one another and to see their own, at least on the first page of Google.
You could see that if you were as your geographic target market in the same position without the Ad Preview tool? Absolutely not. Could you see how your ad compared without this tool, if you do not use Google in the area you were targeting? Of course not.
Here is where the ad preview tool comes in many companies assume their competitors are advertising where they are, but they might be confused - therefore does not have the main function provides the tool. Pretending to be somewhere else and displaying search results from Google as someone else at a different location.

SEMrush

SEMrush is useful for a variety of reasons, but especially when it comes to keyword-level strategies to the analysis of your competitors.
When entering into a particular keyword provides the platform that provide you with the top companies on their respective shows and compares offers to pay organic traffic. The best part? SEMrush shows these results for concrete competitors and it enables you to paste a URL to track down the specific ad creative and landing pages.

Social Media

Nowadays, cross-channel advertising is becoming increasingly popular. Chances are your competitors are optimizing and integrating their advertising campaigns as you are reading this. One way to learn more about your competitors’ strategies is to analyze their social media efforts.
Hashtags, publishing times, channel art, color psychology and the percentage of original versus other people’s content all play a big part when it comes to social media strategies. By keeping track of social media activities, you can compare your own, and draw conclusions to your advantage.

Website Layout And Design

Mobile landing pages are easily overlooked when it comes to landing page analysis, opening up new opportunities for those who are thorough. As mentioned at the Google Partners All-Stars Summit 2014 at Google’s headquarters in Mountain View, California, mobile conversion rates are most influenced by the following five topics:
Homepage and Navigation

Site Search

Commerce and Conversions

Optimizing Form Entry

Form Factor and Usability

Instead of using responsive templates or designs, take the time to create a custom mobile site that incorporates small changes and, in return, generates more conversions. Some beginning changes could include mobile menus, visual calendars, forms above the fold, automatic zoom for form fields and real-time field error notifications.

[…]

The Ruby Rails Aggregator