tag:blogger.com,1999:blog-91500054231090093292024-02-19T16:12:34.301-08:00El Rincon - Marcelo's CornerA bit of code and businessMarcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.comBlogger92125tag:blogger.com,1999:blog-9150005423109009329.post-71882455236011020992015-02-13T09:20:00.000-08:002015-02-13T09:29:06.778-08:00Java SE 8 for the Really Impatient Book ReviewLast year I started working on Java 8, but knew very little of it. I needed a way to dive into the new features, best practices, while working on a project that was going on production with a hard deadline. I decided to grab the <a href="http://www.amazon.com/Java-SE8-Really-Impatient-Course/dp/0321927761/ref=sr_1_1?ie=UTF8&qid=1423847008&sr=8-1&keywords=Java+SE+8+for+the+Really+Impatient" target="_blank">Java SE 8 for the Really Impatient</a> by <a href="http://www.horstmann.com/" target="_blank">Cay S. Horstmann</a>. I put so many tabs on so many great topics/ideas that a picture is well worth it.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6D0CGT_jzktsbLtB6ynBVnVkQ8Rv8n00wyduBrZAfOZp3j4Irp6ghaBXTzadFGiSunRrJQ0xYOwhh-ZklIY1WimrTw6lvO1hbfF25mZEI0fvOGIu8P9NoL-AL4-BkN6oHxGc0ILo5HZs/s1600/IMG_4346.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6D0CGT_jzktsbLtB6ynBVnVkQ8Rv8n00wyduBrZAfOZp3j4Irp6ghaBXTzadFGiSunRrJQ0xYOwhh-ZklIY1WimrTw6lvO1hbfF25mZEI0fvOGIu8P9NoL-AL4-BkN6oHxGc0ILo5HZs/s1600/IMG_4346.JPG" height="320" width="240" /></a></div>
<br />
<br />
<br />
Few things that I picked up from the book is composing <span style="font-family: Courier New, Courier, monospace;">Optional</span> value functions with <span style="font-family: Courier New, Courier, monospace;">flatMap</span><br />
<script class="brush:java" type="syntaxhighlighter">
public class Example {
public static Optional<double> squareRoot(Double x) {
return x < 0 ? Optional.empty() : Optional.of(Math.sqrt(x));
}
public static Optional<Double> revert(Double x) {
return x == 0 ? Optional.empty() : Optional.of(1/x);
}
public static void main(String[] args) {
Double x = 12.33;
Optional<double> result =
Optional.of(x).flatMap(Example::squareRoot)
.flatMap(Example::revert);
System.out.println(result);
x = 0.0;
result = Optional.of(x).flatMap(Example::squareRoot)
.flatMap(Example::revert);
System.out.println(result);
}
}
Optional[0.284785885905587]
Optional.empty
</script>
<br />
The chapter that I focused was on <span style="font-family: Courier New, Courier, monospace;">streams</span>. In my opinion, is the best feature from Java 8. Here, I learned how to transform a stream into a <span style="font-family: Courier New, Courier, monospace;">Map</span> result and some of the issues that you have to keep in mind. For example, if there is more than one element with the same key, the collector will throw an <span style="font-family: Courier New, Courier, monospace;">IllegalStateException</span>.<br />
<script class="brush:java" type="syntaxhighlighter">
public class PersonExample {
public static List<person> getPeopleSample() {
return
Arrays.asList(
new Person("Sara", Gender.FEMALE, 20),
new Person("Sara", Gender.FEMALE, 22),
new Person("Bob", Gender.MALE, 20),
new Person("Paula", Gender.FEMALE, 32),
new Person("Paul", Gender.MALE, 32),
new Person("Jack", Gender.MALE, 2),
new Person("Jack", Gender.MALE, 72),
new Person("Jill", Gender.FEMALE, 12)
);
}
public static void main(String[] args) {
List<person> people = getPeopleSample();
Map<String, Person> map =
people.stream().collect(toMap(Person::getName, Function.identity()));
}
}
</script>
You can override the behavior by supplying the a third function argument that determines the value for the key, given the existing and new value.
<script class="brush:java" type="syntaxhighlighter">
public class PersonExample {
public static List<person> getPeopleSample() {
return
Arrays.asList(
new Person("Sara", Gender.FEMALE, 20),
new Person("Sara", Gender.FEMALE, 22),
new Person("Bob", Gender.MALE, 20),
new Person("Paula", Gender.FEMALE, 32),
new Person("Paul", Gender.MALE, 32),
new Person("Jack", Gender.MALE, 2),
new Person("Jack", Gender.MALE, 72),
new Person("Jill", Gender.FEMALE, 12)
);
}
public static void main(String[] args) {
List<person> people = getPeopleSample();
Map<String, Integer> map =
people.stream()
.collect(toMap(person -> person.getName(),
person -> person.getAge(),
(existingValue, newValue) -> existingValue));
System.out.println(map);
}
}
{Bob=20, Sara=20, Jill=12, Jack=2, Paula=32, Paul=32}<br />
</script>
<br />
Get the book! I'm sure you guys will enjoy it.Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-20208018192677342152015-01-09T12:29:00.001-08:002015-01-09T12:55:17.488-08:00Java 8 Lambdas Book Review<a href="http://shop.oreilly.com/product/0636920030713.do">O'Reily's Java 8 Lambdas</a> by <a href="https://twitter.com/RichardWarburto">Richard Warburn</a> is a "must have" if you are starting with Java 8 or functional programming. The book covers the different features of Java 8, and how using lambdas make you a better programmer. Here are some topics that catched my attention:<br />
<br />
<ul>
<li>Streams - separate the "what" from "how" </li>
<li>Patterns to refactor using lambdas </li>
<li>Higher order functions </li>
<li>Lambda expressions can be used to make many existing design patterns simpler and more readable, especially the command pattern </li>
<li>Lambda-Enabled Concurrency </li>
<li>Lambda-Enabled SOLID principles </li>
</ul>
<br />
<br />
I recommended the book to my team and in my Meetup group - I'm the organizer of the <a href="http://meetup.com/Miami-Java-User-Group/">Miami JVM Meetup</a>. I use the book constantly while doing code reviews. Again, any developer, team leader, or architect who is starting to use Java 8 should get this book. I really enjoyed the ending and finally understood the Reactive Programming approach. As the author mentioned,
<br />
<blockquote>
The critical design tool for software development is a mind well educated in design principles. It is not...technology. - Craig Larman</blockquote>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-69811091308177267972014-12-29T07:29:00.001-08:002014-12-29T07:49:47.627-08:00BDD in Action - Book ReviewFinished reading <a href="http://manning.com/smart/" target="_blank">Manning's BDD in Action</a> (behavior-driven development) by <a href="https://twitter.com/wakaleo" target="_blank">John Ferguson Smart</a> which I found it very insightful. There are four things that struck me about this book:<br />
<br />
<ol>
<li>Don't write unit tests, write low-level specifications</li>
<li>Favor outside-in development</li>
<li>Learned about Spock framework</li>
<li>There is a difference between a story and a feature</li>
</ol>
<br />
<br />
I used <a href="https://github.com/rspec/rspec-rails" target="_blank">RSpect</a> before but without a clear understanding of BDD, so I wrote unit tests (test scripts) rather than low-level specifications. The book explains why BDD is important along with details steps and examples. BDD is when we write <i>behavior and specification</i> that then drive the software. One of the key goals of BDD is to ensure that everyone has a clear understanding of what the project is trying to deliver, and of the underlying business objective of the problem. BDD is TDD but with better guidelines or even total new approach to developing. This is why wording and semantics are important: the tests need to clearly explains the business behavior they're demonstrating. It encourages people to write tests in terms of the expectations of the program's behavior in a given set of circumstances.<br />
<br />
When writing user stories we were told to use this template:<br />
<span style="font-family: Arial, Helvetica, sans-serif;">As a </span><span style="font-family: Courier New, Courier, monospace;"><stakeholder></span><span style="font-family: Arial, Helvetica, sans-serif;"> I want </span><span style="font-family: Courier New, Courier, monospace;"><something></span><span style="font-family: Arial, Helvetica, sans-serif;"> so that I can </span><span style="font-family: Courier New, Courier, monospace;"><achieve some business goal></span><span style="font-family: Arial, Helvetica, sans-serif;">.</span><br />
<br />
Once you have a story, then you have to explore the details by asking the users and other stakeholders for concrete examples.<br />
<br />
In BDD the following notation is often used to express examples:<br />
<b>Given</b> <a context>: describes the preconditions for the scenario and prepare the test environment<br />
<b>When</b> <something happens>: describes the action under the test.<br />
<b>Then</b> <you expect some outcome><br />
<br />
Example:<br />
<b>Story</b>: Returns go to stock<br />
<br />
In order to keep track of stock<br />
As a store owner<br />
I want to add items back to stock when they're returned<br />
<br />
<b><u>Scenario 1</u></b>: Refunded items should be returned to stock<br />
<span style="font-family: Courier New, Courier, monospace;"> <b>Given</b> a customer previously bought a black sweater from me</span><br />
<span style="font-family: Courier New, Courier, monospace;"> And I currently have three black sweaters left in stock</span><br />
<span style="font-family: Courier New, Courier, monospace;"> <b>When</b> he returns the sweater for a refund</span><br />
<span style="font-family: Courier New, Courier, monospace;"> <b>Then</b> I should have four black sweaters in stock</span><br />
<br />
<u><b>Scenario 2</b></u>: Replaced items should be returned to stock<br />
<span style="font-family: Courier New, Courier, monospace;"> <b>Given</b> that a customer buys a blue garment</span><br />
<span style="font-family: Courier New, Courier, monospace;"> And I have two blue garments in stock</span><br />
<span style="font-family: Courier New, Courier, monospace;"> And three black garments in stock.</span><br />
<span style="font-family: Courier New, Courier, monospace;"> <b>When</b> he returns the garment for a replacement in black,</span><br />
<span style="font-family: Courier New, Courier, monospace;"> <b>Then</b> I should have three blue garments in stock</span><br />
<span style="font-family: Courier New, Courier, monospace;"> And two black garments in stock</span><br />
<br />
As the book <a href="http://www.manning.com/adzic/" target="_blank">Specification by Example</a> mentioned, instead of waiting for specifications to be expressed precisely for the first time in the implementation, successful teams illustrate specifications using examples. The team works with the business user or domain experts to identify key examples that describes the functionality. During this process, developers and testers often suggest additional examples that illustrate the edge cases or address areas of the system that are particular problematic. This flushes out functional gaps and inconsistencies and ensure that everyone involved has a share understanding of what needs to be delivered, avoid rework that results from misinterpretation and translation.<br />
<br />
Besides understanding the difference of unit tests and specifications, the book also talks about the difference of features vs. user stores. They are <b>NOT</b> the same. A feature is a functionality that you deliver to the end users or to the other stakeholders to support a capability that they need in order to achieve their business goals. A user story is a planning tool that helps you flesh out details of what you need to deliver for a particular feature. You can have features without having stories. Is a matter of fact, a good practice is to summarize the "Given When" sections of the scenario in the title and avoid including any expected outcomes. Because scenarios are based on real business examples, the context and events are usually stable, but the expected outcome may change as the organization changes and evolves the way it does business.<br />
<br />
Besides the language syntax, I discovered the <a href="https://code.google.com/p/spock/" target="_blank">Spock</a> framework. It lets you write concise and descriptive tests with less boiler plate code than would be needed using java. The syntax encourages people to write tests in terms of your expectations of the program's behaviors in a given set of circumstances.<br />
<br />
Example:<br />
<script class="brush:java" type="syntaxhighlighter">
class HelloSpock extends Specification {
def "length of Spock's and his friends' names"() {
expect:
name.size() == length
where:
name | length
"Spock" | 5
"Kirk" | 4
"Scotty" | 6
}
}
</script>
<br />
<br />
While I was reading this book, two quotes came to my head:<br />
<br />
<blockquote class="tr_bq">
Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live - M. Goldin.</blockquote>
<br />
<blockquote class="tr_bq">
Programs must be written for people to read, and only incidentally for machines to execute - Adbson and Sussman.</blockquote>
<br />
The other insightful thing that I learned is that BDD favors an outside-in development approach. Which they include:<br />
<br />
<ul>
<li>Start with a high-level acceptance criterion that you want to implement</li>
<li>Automate the acceptance criterion as pending scenarios, breaking the acceptance criterion into smaller steps</li>
<li>Implement the acceptance criterion step definition, imagining the code you'd like to have to make each step work</li>
<li>Use these step definition to flesh out unit tests that specify how the application code will behave</li>
<li>Implement the application code, and refactor as required</li>
</ul>
<br />
<br />
There are many benefits of outside-in development, but the principle motivations are summarized here:<br />
<br />
<ul>
<li>Outside-in code focuses on business value</li>
<li>Outside-in code encourages well-designed, easy to understand code</li>
<li>Outside-in code avoid waste</li>
</ul>
<br />
<br />
As I mentioned, I enjoy the book and I found it very insightful. Some (if not all) the ideas of the book has been around for decades. I believe that this book is great for an architect, programmer, testers, project manager, product owner, and scrum masters.Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-5001290494615320802014-11-10T18:16:00.000-08:002014-11-10T18:16:50.197-08:00Validate Map Collections via MatchersI was introduced to <a href="https://code.google.com/p/hamcrest/wiki/Tutorial">Hamcrest Matchers</a> by the 3C team. I am really liking it. Today, I stumbled on a validation of a Map. Here is how I usually solved the problem and then how I solved it using the syntax sugar of Matchers.<br />
<br />
<script class="brush:java" type="syntaxhighlighter">
@Test
public void shouldMatchEveryItemOnBothMaps() {
Map<String, String> entries = new HashMap<>();
entries.put("key1", "value1");
entries.put("key2", "value2");
entries.put("key3", "value3");
entries.put("key4", "value4");
Map<String, String> mapShouldMatch = ImmutableMap.of("key1", "value1","key2", "value2"
,"key3", "value3","key4", "value4");
//validate that the size match
assertThat(entries.size(), is(mapShouldMatch.size()));
for (Map.Entry<String, String> entry : entries.entrySet()) {
assertThat(entries.get(entry), is(mapShouldMatch.get(entry)));
}
}
</script>
<script class="brush:java" type="syntaxhighlighter">
@Test
public void shouldMatchEveryItemOnBothMapsUsingMatcher() {
Map<String, String> entries = new HashMap<>();
entries.put("key1", "value1");
entries.put("key2", "value2");
entries.put("key3", "value3");
entries.put("key4", "value4");
assertThat(entries, allOf(hasSize(4),
(Matcher) hasEntry("key1", "value1"),
(Matcher) hasEntry("key2", "value2"),
(Matcher) hasEntry("key3", "value3"),
(Matcher) hasEntry("key4", "value4")));
}
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-21726980975973964182014-11-09T15:41:00.002-08:002014-11-10T04:27:46.605-08:00Lambdas and Java 8Java 1.8 introduces the concept of streams, which are similar to iterators.<br />
<br />
Why Lambdas are good for you:<br />
<ul>
<li>Form the basis of functional programming language</li>
<li>Make parallel programming easier </li>
<li>Write more compact code </li>
<li>Richer data structure collections </li>
<li>Develop cleaner APIs </li>
</ul>
Lambdas Expression Lifecycle - think of them as having a two stage lifecycle:<br />
<ol>
<li>Convert the lambda expression to a function.</li>
<li>Call the general function</li>
</ol>
<br />
<script class="brush:java" type="syntaxhighlighter">
x -> System.out.println(x);
//same a calling this function:
public static void generatedNameOfLambdaFunction(Integer x) {
System.out.println(x);
}
</script>
Streams have two types of operations: intermediate and terminal.<br />
<b>Intermediate operation</b>: specifies tasks to perform on the stream's elements and always results in a new stream.<br />
<span style="font-family: Courier New, Courier, monospace;">filter</span>: Result in a stream containing only the elements that satisfy a condition.<br />
<span style="font-family: Courier New, Courier, monospace;">distinct</span>: Result in a stream containing only the unique element.<br />
<span style="font-family: Courier New, Courier, monospace;">limit</span>: Result in a stream with the specified number of elements from the beginning of the original stream.<br />
<span style="font-family: Courier New, Courier, monospace;">map</span>: Result in a stream in which each element of the original stream is mapped to a new value (possibly of a different type).<br />
<span style="font-family: Courier New, Courier, monospace;">sorted</span>: Result in a stream in which the elements are in sorted order. The new stream has the same number of elements as the original stream.<br />
<br />
<br />
<b>Terminal operations</b> initiates processing of a stream pipeline's intermediate operations and produces results.<br />
<span style="font-family: Courier New, Courier, monospace;">forEach</span>: Performs processing on every element in a stream.<br />
<span style="font-family: Courier New, Courier, monospace;">average</span>: Calculates the average of the elements in a numeric stream.<br />
<span style="font-family: Courier New, Courier, monospace;">count</span>: Returns the number of elements in the stream.<br />
<span style="font-family: Courier New, Courier, monospace;">max</span>: Locates the largest value in a numeric stream.<br />
<span style="font-family: Courier New, Courier, monospace;">min</span>: Locates the smallest value in a numeric stream.<br />
<span style="font-family: Courier New, Courier, monospace;">reduce</span>: Reduces the element of a collection to a single value using an associative accumulation function (e.g. a lambda that adds two elements -- in Scala this is the "map" operator).<br />
<br />
<br />
Mutable reduction operations: creates a container (such as a collection or StringBuilder)<br />
<span style="font-family: Courier New, Courier, monospace;">collect</span>: Creates a new collection of elements containing the results of the streams's prior operations.<br />
<span style="font-family: Courier New, Courier, monospace;">toArray</span>: Creates an array containing the results of the stream's prior operations.<br />
<br />
<br />
Search operations<br />
<span style="font-family: Courier New, Courier, monospace;">findFirst</span>: Find the first stream element based on the prior intermediate operations; immediately terminates the processing of the stream pipeline once such an element is found.<br />
<span style="font-family: Courier New, Courier, monospace;">findAny</span>: Finds any stream element based on the prior intermediate operations: immediately processing of the stream pipeline once such an element is found.<br />
<span style="font-family: Courier New, Courier, monospace;">anyMatch</span>: Determines whether any stream elements match a specified conditions; immediately terminates processing of the stream pipeline if an element matches.<br />
<span style="font-family: Courier New, Courier, monospace;">allMatch</span>: Determines whether all of the elements in the stream match a specified condition.<br />
<span style="font-size: large;"><br /></span>
<span style="font-size: large;">Examples:</span><br />
<script class="brush:java" type="syntaxhighlighter">
public class StreamsExampleTest {
@Test
public void shouldReturnDifferentMathAggregations() {
int [] values = {3,10,6,1,4,8,2,5,9,7};
//Display all the values
IntStream.of(values).forEach(value -> System.out.printf("%d ", value));
assertThat(values.length, is(10));
//Count all values
assertThat(IntStream.of(values).sum(), is(55));
assertThat(IntStream.of(values).min().getAsInt(), is(1));
int [] noValues = {};
assertThat(IntStream.of(noValues).min().orElse(-1), is(-1));
assertThat(IntStream.of(values).average().getAsDouble(), is(5.5));
//Sum via reduce - fold (Scala)
assertThat(IntStream.of(values).reduce(0, (x, y) -> x + y), is(55));
//Product of values
assertThat(IntStream.of(values).reduce(1, (x, y) -> x * y), is(3628800));
//Sum of squares
assertThat(IntStream.of(values).reduce(0, (x, y) -> x + y * y), is(385));
//Even values displayed in sorted order
System.out.printf("%nEven values displayed in sorted order: ");
IntStream.of(values)
.filter(value -> value % 2 == 0)
.sorted()
.forEach(value -> System.out.printf("%d ", value));
//Odd values multiplied by 10
System.out.printf("%nOdd values multiplied by 10 displayed in sorted order: ");
IntStream.of(values)
.filter(value -> value % 2 != 0)
.map(value -> value * 10)
.sorted()
.forEach(value -> System.out.printf("%d ", value));
//Sum range integers from 1 to 10, exclusive - not including the second element (10)
assertThat(IntStream.range(1, 10).sum(), is(45));
//Sum range integers from 1 to 10, inclusive. Include first and second argument
assertThat(IntStream.rangeClosed(1, 10).sum(), is(55));
}
@Test
public void shouldReturnStreamArrayAndMathAggregation() {
Integer [] values = {2, 9, 5, 0, 3, 7, 1, 4, 8, 6};
assertThat(Arrays.asList(values).toString(),
is("[2, 9, 5, 0, 3, 7, 1, 4, 8, 6]"));
assertThat(Arrays.stream(values).sorted().collect(Collectors.toList()).toString(),
is("[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"));
//Greater than four
List<integer> greaterThanFour = Arrays.stream(values)
.filter(value -> value > 4)
.collect(Collectors.toList());
assertThat(greaterThanFour.toString(), is("[9, 5, 7, 8, 6]"));
assertThat(Arrays.stream(values)
.filter(value -> value > 4)
.sorted()
.collect(Collectors.toList()).toString(),
is("[5, 6, 7, 8, 9]"));
assertThat(greaterThanFour.stream()
.sorted()
.collect(Collectors.toList()).toString(),
is("[5, 6, 7, 8, 9]"));
}
@Test
public void shouldValidateStream() {
String [] strings = {"Red", "orange", "Yellow", "green", "Blue", "indigo", "Violet"};
List<string> upperCaseList =
Arrays.stream(strings).map(String::toUpperCase).collect(Collectors.toList());
assertThat(upperCaseList, hasItems("RED", "ORANGE", "YELLOW", "GREEN", "BLUE", "INDIGO",
"VIOLET"));
List<string>
sortedList = Arrays.stream(strings)
.filter(s -> s.compareToIgnoreCase("n") < 0)
.sorted(String.CASE_INSENSITIVE_ORDER).collect(Collectors.toList());
String [] shouldBeSortedAsSuch = {"Blue", "green", "indigo"};
assertThat(sortedList, is(Arrays.asList(shouldBeSortedAsSuch)));
}
@Test
public void collectEveryWordAndMakeItUpperCase() {
String [] words = {"a", "b", "hello"};
List<String> test = Arrays.asList(words);
List<string> listOfWordSizes = test.stream().map(word -> word.toUpperCase()).collect(Collectors.toList());
assertThat(listOfWordSizes, hasItems("A", "B", "HELLO"));
}
}
</script>
<br />
<h4>
Refactor: </h4>
This has done so many different changes to some of my code. Here are some example of before and after:
Before:
<script class="brush:java" type="syntaxhighlighter">
@Component
class BookingCommandLineRunner implements CommandLineRunner {
private BookingRepository bookingRepository;
@Autowired
BookingCommandLineRunner(BookingRepository bookingRepository) {
this.bookingRepository = bookingRepository;
}
@Override
public void run(String... args) throws Exception {
for(Booking booking : bookingRepository.findAll()) {
System.err.println(booking);
}
}
}
</script>
I wanted to printout some of the results and so I leveraged Spring's <a href="http://docs.spring.io/autorepo/docs/spring-boot/1.1.5.RELEASE/api/org/springframework/boot/CommandLineRunner.html">CommandLineRunner</a>.<br />
<br />
Here's the after code:
<script class="brush:java" type="syntaxhighlighter">
@Bean
CommandLineRunner commandLineRunner(BookingRepository bookingRepository) {
return (String... args) -> {
bookingRepository.findAll().forEach(System.err::println);
bookingRepository.findByBookingName("Marcelo").forEach(System.err::println);
};
}
</script>
Another example fetching a collection of records from a database:
<script class="brush:java" type="syntaxhighlighter">
JdbcTemplate template = getTemplate();
List<product> products = template.query("SELECT * from PRODUCTS",
new RowMapper<product>() {
@Override
public Product mapRow(ResultSet rs, int rowNum) throws SQLException {
Integer id = rs.getInt("id")
String description = rs.getString("description");
Integer quantity = rs.getInt("quantity");
BigDecimal price = rs.getBigDecimal("price");
Date availability = rs.getDate("available_date");
Product product = new Product(id, description, quantity, price, availability);
return product;
}
});
</script>
I can refactored it doing this:
<script class="brush:java" type="syntaxhighlighter">
JdbcTemplate template = getTemplate();
List<product> products = template.query("SELECT * from PRODUCTS", (ResultSet rs, int rowNum) -> {
Integer id = rs.getInt("id")
String description = rs.getString("description");
Integer quantity = rs.getInt("quantity");
BigDecimal price = rs.getBigDecimal("price");
Date availability = rs.getDate("available_date");
Product product = new Product(id, description, quantity, price, availability);
return product;
}
});
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-20832559682708619642014-10-16T16:48:00.002-07:002014-10-16T16:49:43.000-07:00Changing Java SDK in IntelliJ IDEA 13We just migrated to Java 1.8. In my personal computer, I installed the JDK 1.8 and make sure that Maven was running fine in using the latest Java version. I'm using a Mac, so when I ran it in Terminal everything worked. However, when I ran it in my <a href="https://www.jetbrains.com/idea/">IntelliJ IDEA</a>, it said that it was running Java 1.7.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBj5IQvqlp55YbcokQrDr9-EvBnertTnGA_7tZXquUz0tGvpbXL0VSjAWbH1_dsmhgdemHDTdbX5enSE6OQpsSdc4KRDjNptCh6unTyOQiEaGRPz-SBoFbOrLLxRsW5PzWKyha4jgX840/s1600/java-and-mercury.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBj5IQvqlp55YbcokQrDr9-EvBnertTnGA_7tZXquUz0tGvpbXL0VSjAWbH1_dsmhgdemHDTdbX5enSE6OQpsSdc4KRDjNptCh6unTyOQiEaGRPz-SBoFbOrLLxRsW5PzWKyha4jgX840/s1600/java-and-mercury.png" /></a></div>
<br />
To changed it, go to the following menu: <b>File, Project Structure, then click on Project</b>. <br />
Here is where you can set the SDK for your project. Just change it to the right SDK and that's it.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRB5p0RowUmJHS61VQ11WRK3bB9nzSqSdos-8fOLgKwM-O0BRYZ4WwoOEpt239TGhgglFFOTPHYtoYFkwXfb6pcCENRPCw-RAeR2PyKEQmcU-ETp3eYE8Uy2gOJ6VhfO9KvEJ65OrCpp4/s1600/Project_Structure.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRB5p0RowUmJHS61VQ11WRK3bB9nzSqSdos-8fOLgKwM-O0BRYZ4WwoOEpt239TGhgglFFOTPHYtoYFkwXfb6pcCENRPCw-RAeR2PyKEQmcU-ETp3eYE8Uy2gOJ6VhfO9KvEJ65OrCpp4/s1600/Project_Structure.png" height="205" width="640" /></a></div>
<br />
Happy coding.Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-30475392567254262322014-09-03T14:27:00.001-07:002014-10-06T08:42:37.591-07:00Learning CassandraI just finished reading <a href="http://www.amazon.com/Practical-Cassandra-Developers-Addison-Wesley-Analytics/dp/032193394X/ref=sr_1_1?ie=UTF8&qid=1411572219&sr=8-1&keywords=practical+cassandra" target="_blank">Practical Cassandra</a>. I enjoyed this book and it helped with my presentation at <a href="http://www.rokk3rlabs.com/" target="_blank">Rokk3r Labs</a> in Miami Beach. You can tell that <a href="https://twitter.com/devdazed" target="_blank">Russell Bradberry</a> and <a href="https://twitter.com/elubow" target="_blank">Eric Lubow</a> spent sometime thinking about this book. I like that it's straight to the point for a developer, but it is also useful for sysadmins and managers. I enjoyed the troubleshooting and "use cases".<br />
<br />
The book mentions, "where Cassandra fits in". This is a question that I constantly get when talking about Cassandra. Many people want to know, "why not [NoSQL database of your choice]?". The short answer is: if you want fast writes, multi-data center support baked into your system, a truly scalable system with tons of metric, then you should consider Cassandra. However, I always follow my answer by saying that the best way to know if Cassandra fits into the role, is to understand it. When I started using it, I had to stop myself thinking about all that I know about data modeling with RDBMS. Most of the stuff that we learned in RDBMS is actually an anti-pattern for Cassandra - normalization, build your model first, index with high-cardinality, leverage joints. Don't think of a relational table, think of a nested, sorted map data structure.<br />
<br />
<h3>
Tunable Consistency and Polyglot Databases</h3>
Many people don't understand that you can tune the consistency of Cassandra. The followings are the configuration that you can have for reads and writes:<br />
<br />
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">ANY</span>: is for writes only and ensures that the write will persists on any server in the cluster.</li>
<li><span style="font-family: Courier New, Courier, monospace;">ONE</span>: ensures that at lease one server within the replicate set with persist the write or respond to the read</li>
<li><span style="font-family: Courier New, Courier, monospace;">QUORUM</span>: means the read/write will go to the half of the nodes in the replica set plus one.</li>
<li><span style="font-family: Courier New, Courier, monospace;">LOCAL_QUORUM</span>: it's just like "quorum" except that it is only for the nodes in that data center.</li>
<li><span style="font-family: Courier New, Courier, monospace;">EACH_QUORUM</span>: is like "quorum" but ensures a quorum read/write on each of the data centers.</li>
<li><span style="font-family: Courier New, Courier, monospace;">ALL</span>: ensures that all nodes in a replica set will receive the read/write.</li>
</ul>
<br />
One of the things Cassandra does not do, is joins or ad-hoc queries. This is a something that Cassandra simply doesn't do and other tools do it better (Solr, ElasticSearch, etc). This is what people are calling to Polyglot Data.<br />
<br />
<h3>
Gossip vs Snitch</h3>
Practical Cassandra helped me understand the difference between the "gossip" and "snitch" protocol. This is something that I struggled time and time again. Gossip is the protocol that Cassandra uses to discover information about the new nodes. When bringing a new node into the cluster, you must specify a "seed node". The seed nodes are a set of nodes that are used to given information about the cluster to newly joining nodes. As you can imagine, the seed nodes should be stable and should point to other seed nodes.<br />
<br />
The snitch protocol helps map IPs to racks and data centers. It creates a topology by grouping nodes together to help determine where data is read from. There are few types of snitches: simple, dynamic, rack interfering, EC2, and Ec2MultiRegion. <br />
<br />
<br />
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">Simple</span> snitch is recommended for a simple cluster (one datacenter as one zone in a cloud architecture).</li>
<li><span style="font-family: Courier New, Courier, monospace;">Dynamic</span> snitch wraps over the SimpleSnitch and provides an additional adaptive layer for determining the best possible read location.</li>
<li><span style="font-family: Courier New, Courier, monospace;">RackInferringSnitch</span> works by assuming it knows the topology of your network, by the ocftets in node's IP address.</li>
<li><span style="font-family: Courier New, Courier, monospace;">EC2Snith</span> snitch EC2 Snitch is for Amazon Web Service (AWS)-based deployments where the cluster sits within a single region.</li>
<li><span style="font-family: Courier New, Courier, monospace;">EC2MultiRegionSnitch</span> is for AWS deployments where the Cassandra cluster spans multiple regions.</li>
</ul>
<br />
<h3>
</h3>
<h3>
Node Layout</h3>
Prior to Cassandra 1.2, one token was assigned to each node. Whenever you had a node that would have a lot of load of data, that would be consider a "hot spot". Most of the times, you will just add another node to leverage the "hot spot", but then you had the "rebalance" the cluster. Virtual nodes or vnodes, provide a Cassandra node with the ability to be responsible for many token ranges. Within a cluster, they can be noncontiguous and selected at random. This provide a greater distribution of data than the non-vnode paradigm.<br />
<br />
<h3>
Performance</h3>
The performance chapter was also another very interesting chapter. Being a developer it introduced me to common *nix tools like <span style="font-family: Courier New, Courier, monospace;">vmstat</span>, <span style="font-family: Courier New, Courier, monospace;">iostat</span>, <span style="font-family: Courier New, Courier, monospace;">dstst</span>, <span style="font-family: Courier New, Courier, monospace;">htop</span>, <span style="font-family: Courier New, Courier, monospace;">atop</span>, and <span style="font-family: Courier New, Courier, monospace;">top</span>. All of these tools provide a picture of usage. It also explained how instrumentations goes a long way. Also, if one node becomes too slow to respond, the <span style="font-family: Courier New, Courier, monospace;">FailureDector</span> will remove it.<br />
<br />
An easy optimization for Cassandra is putting your <span style="font-family: Courier New, Courier, monospace;">CommitLog</span> directory on a separate drive from your data directories. <span style="font-family: Courier New, Courier, monospace;">CommitLog</span> segments are written to every time a <span style="font-family: Courier New, Courier, monospace;">MemTable</span> is flushed to disk. You can do this setting in the <span style="font-family: Courier New, Courier, monospace;">cassandra.yml</span> by setting the <span style="font-family: Courier New, Courier, monospace;">data_directory</span> and <span style="font-family: Courier New, Courier, monospace;">commitlog_directory</span>.<br />
<br />
<h3>
Metrics</h3>
Cassandra goes out of her ways to provide lots of metrics. With all these metrics you can do capacity planning. Once you start getting all these metrics, you'll be able to differentiate trends and be able to proactively add or remove nodes. For example, you can monitor the <span style="font-family: Courier New, Courier, monospace;">PendingTask</span> under the <span style="font-family: Courier New, Courier, monospace;">CompactionManagerMBean</span> to know the speed and volume with which you can ingest data, you will need to find a comfortable set of threshold for your system. Another example is to monitor the high request latency, which can indicate that there is a bad disk or that your current read pattern is starting to slow down.<br />
<br />
These are some of the metrics that you can get via JMX:<br />
<br />
<br />
<ul>
<li>DB: monitors the data storage of Cassandra. You can monitor the cache and the <span style="font-family: Courier New, Courier, monospace;">CommitLogs</span>, or even information about the <span style="font-family: Courier New, Courier, monospace;">ColumnFamily</span>.</li>
<li>Internal: these cover the state and statistics around the staged architecture (gossip information and hinted handoffs).</li>
<li>Metrics: these are client request metrics (timeouts and "unavailable" errors).</li>
<li>Net: these metrics monitored the network (failure detector, gossiper, messaging service, and streaming service).</li>
<li>Request: these are metrics about request from the client (read, write, and replication).</li>
</ul>
<br />
<br />
There are still a lot of stuff that I need to learn about Cassandra. Specially about the data model. It's very tricky to start thinking about your queries (pre-optimized queries like Nate McCall calls them). In all, the book does covers the basics .Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-68183857815700962382014-07-11T15:49:00.001-07:002014-07-13T16:41:19.533-07:00Case for Code ReviewsI have been doing code reviews for about six months at <a href="http://www.3cinteractive.com/" target="_blank">3CInteractive</a>. Since I'm so new at it, it's hard for me blog about "best practices" or even do a presentation on code reviews. Therefore, I decided to have an <a href="http://www.meetup.com/Miami-Java-User-Group/events/161811502/" target="_blank">open space meetup at our Miami JVM Group</a>. The open space will be around "effective code review process". I'm hoping to learn about the following: Who is using it? Who are not using it and why? Who thinks they're not useful and why? What have they learned? What were some of their challenges? This post is based on my experience and some of my lessons learned while working with internal and offshore teams.<br />
<br />
<h3>
Benefits of Code Reviews</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJNTVTg_82YitvpsAWVrdSKKdUeAzKSjDS4g_P9FFkQMKkc4Gu23VwzsPx9TgFYXDEGrOspz22PrlDx1upS0LuJJh_-4XqJ29CnK2vdW0vzMkC_EOI58icHy5T0_s_1wTOBsXJQF5sQy0/s1600/Corbis-42-51651787.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJNTVTg_82YitvpsAWVrdSKKdUeAzKSjDS4g_P9FFkQMKkc4Gu23VwzsPx9TgFYXDEGrOspz22PrlDx1upS0LuJJh_-4XqJ29CnK2vdW0vzMkC_EOI58icHy5T0_s_1wTOBsXJQF5sQy0/s1600/Corbis-42-51651787.jpg" height="320" width="213" /></a></div>
<div>
<br /></div>
<ul>
<li>You write better code when you know it will be reviewed.</li>
<li>A second (or third, or fourth) set of eyes will help spot defects. This is very similar to pair programming, but it works even better if you're working with an offshore team. It's also a great way of learning new APIs. For example, someone could tell me that my code can be easily done using the <a href="https://code.google.com/p/guava-libraries/" target="_blank">Guava</a> library, or that the code is actually an "aggregator" Enterprise Implementation Pattern, and I should probably look at <a href="http://camel.apache.org/" target="_blank">Camel</a>.</li>
<li>More than one person understand your code (cross-pollination or avoid silos). Having more than one person look at your code helps spread the knowledge and context of the problem or solution.</li>
<li>Reducing the learning curve for new developers. I believe that even junior developers should be part of the code review process. It's a great way for them to learn about the code base and they become more productive.</li>
</ul>
<h2>
</h2>
<h3>
</h3>
<h3>
</h3>
<h3>
What To Look For</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIiwHzFqwdsyuYIASbeGcZtHYhBqAQduqg54Ah8mNf80zQZ2UxO953MGMK5Pz315nPRnet0dZIIJUA8ibRx4ZKBqymdGUoK4gMDZqjnh8UaB51PKJ6yeZmKa6ZqyXKcaPJX7jO2jJYGrs/s1600/Corbis-42-58055539.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgIiwHzFqwdsyuYIASbeGcZtHYhBqAQduqg54Ah8mNf80zQZ2UxO953MGMK5Pz315nPRnet0dZIIJUA8ibRx4ZKBqymdGUoK4gMDZqjnh8UaB51PKJ6yeZmKa6ZqyXKcaPJX7jO2jJYGrs/s1600/Corbis-42-58055539.jpg" height="213" width="320" /></a></div>
<div>
<br /></div>
<ul>
<li>Bad design. Highlight issues such as SQL injection, look out for lack of design patterns, or anti-patterns. Things like separations of concern, encapsulation, and apply certain basic OOD principles - DRY, encapsulates what varies, open-close principle, etc.</li>
<li>Performance hazard. For example, memory leaks.</li>
<li>Lack of clarity - the application should work and the code should be readable. For example, a class named "SomethingServiceImpl" with no documentation on the class will be highlighted and will prompt a change request to the developer. Also, a big nested if statement that is not quiet clear will prompt a change request. </li>
<li>Not consistent or not according to standards. Having a set of standards makes code reviews a lot simpler. It also sets a norms for the team. For example, not using <a href="http://en.wikipedia.org/wiki/Domain-driven_design" target="_blank">Domain-Drive Development</a> standards, consolidating APIs (Guava vs Jakarta Commons), having a handful of languages, and having a code style rules.</li>
</ul>
<h2>
</h2>
<h3>
</h3>
<h3>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkF3mAN2ajIREURU0M1UHyUx68KSST0nFwJjgsIRmlIZG7M9whSq6005AanTiXNB-gS_kHfzvaaGrrdF8xpg6hbgTcnQf2refStVEhF6vrkBgUcfNTpguM1tqlbtenXgX4g6m2N3MVmZ8/s1600/Corbis-42-22343746.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkF3mAN2ajIREURU0M1UHyUx68KSST0nFwJjgsIRmlIZG7M9whSq6005AanTiXNB-gS_kHfzvaaGrrdF8xpg6hbgTcnQf2refStVEhF6vrkBgUcfNTpguM1tqlbtenXgX4g6m2N3MVmZ8/s1600/Corbis-42-22343746.jpg" height="240" width="320" /></a></h3>
<h3>
What Not To Look For</h3>
<div>
<br /></div>
<div>
<ul>
<li>Premature optimization. Don't try to optimize all of it at once. As my buddy Tyler mentions, "make it work, then make it better".</li>
<li>Skills and expertise gaps. In our company, we allow all developer to do code reviews, including junior developers. These developers gain a lot of knowledge about doing code reviews.</li>
<li>Personal style. If the CTO goes over and says, "I wouldn't do it like that" it bring noise to the process.</li>
</ul>
</div>
<br />
<h3>
Quality and Dissemination of Knowledge </h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAf69lv8X3YJWFBFibUkZ2OZ5bsd5vQD0mnddAi3CG5J0rZt8QjXRZV7iq09jqOvIkeQmveW1WNC7JSIwWdLMfJ-rIddPE7d8Nj9OABXJVtUV1o7_IIvfTlTl45vFuxDmWk-BUpyIosQU/s1600/Corbis-42-59311093.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAf69lv8X3YJWFBFibUkZ2OZ5bsd5vQD0mnddAi3CG5J0rZt8QjXRZV7iq09jqOvIkeQmveW1WNC7JSIwWdLMfJ-rIddPE7d8Nj9OABXJVtUV1o7_IIvfTlTl45vFuxDmWk-BUpyIosQU/s1600/Corbis-42-59311093.jpg" height="213" width="320" /></a></div>
My team is responsible for improving code quality, lower defects in code, improve communication about code content, and teaching and mentoring of junior developers. Code reviews has helped us on shorter development cycles, more customer satisfaction, and more maintainable code. But most important, it has help us spread the knowledge and norms. As per the book <a href="http://www.amazon.com/Things-Every-Software-Architect-Should/dp/059652269X/ref=sr_1_1?ie=UTF8&qid=1405117692&sr=8-1&keywords=97+things+every+architect+should+know" target="_blank">97 Things Every Software Architect Should Know</a>,<br />
<blockquote class="tr_bq">
Chances are your biggest problem isn't technical</blockquote>
Most projects are built by people, and those people are the foundation for success and failure. So, it pays to think about what it takes to help make those people successful.Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-71359333826602517102014-03-25T16:02:00.000-07:002014-03-25T16:02:05.464-07:00Find Type of Vagrant VM is RunningI am really enjoying Vagrant. It's one of those tools that are indispensable. However, today I wanted to install a CentOS VM in my application and I didn't remember the version name that I was using in my other VMs. To find out, the only thing that you have to do is to check a previous VM. Here's an example:<br />
vim ~/vagrant_boxes/kafka/Vagrantfile<br />
You will be able to see the version inside the file:<br />
<br />
<script class="brush:java" type="syntaxhighlighter">
-*- mode: ruby -*-
# vi: set ft=ruby :
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# All Vagrant configuration is done here. The most common configuration
# options are documented and commented below. For a complete reference,
# please see the online documentation at vagrantup.com.
# Every Vagrant virtual environment requires a box to build off of.
config.vm.box = "centos6.4"
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-64721422051516517212014-03-21T09:27:00.000-07:002014-04-04T06:38:31.660-07:00Strata 2014 - Newbie PerspectiveMarc Andreessen noticed that <a href="http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460" target="_blank">software is eating the world</a>. I see the same thing with Big Data. Big Data is shaping the world around us. It has been used on presidential elections, weather reports, consumer analysis/sentiment, fraud check, etc. <a href="http://strataconf.com/" target="_blank">Strata</a> conference is the epicenter of new technologies, use cases, and new innovations related to Big Data. I've been meaning to go there for quite some time. Previously, I purchased the videos from <a href="http://www.oreilly.com/" target="_blank">O'Reilly </a>because I couldn't make it. Thanks to my current company, <a href="http://www.3cinteractive.com/" target="_blank">3C</a> (they're pretty awesome), I was able to go along with five of my coworkers. It's the place where you can meet the experts, the main committers, and ask them questions. If your eyes get dilated when you talk of Hadoop, or you get exited when you need to solve a problem that has to do with a huge amount of data including the famous "three V's" (volume, velocity, and variety), then this conference is for you. This is a quick summary of my experience of the conference. <br />
<br />
The conference revolved around four clusters:<br />
<br />
<ol>
<li>How quickly can you get the data into your system (ingest)</li>
<li>How fast can you show the results</li>
<li>It's all about presentation (charts)</li>
<li>Big Data doesn't mean Hadoop</li>
</ol>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgI83zIkW77Gc1tnk4foCk-rQx5J33jcNJqcNZciyWtCdZ4eS0t18giowSb-wAVAkdJ1gycZ60FSmx792UdO6X-fspz6oD_SYLdZMFpurzHFUW_5SoBtg5KkyMght7cMiOkDkrubyMNBBA/s1600/strata1.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgI83zIkW77Gc1tnk4foCk-rQx5J33jcNJqcNZciyWtCdZ4eS0t18giowSb-wAVAkdJ1gycZ60FSmx792UdO6X-fspz6oD_SYLdZMFpurzHFUW_5SoBtg5KkyMght7cMiOkDkrubyMNBBA/s1600/strata1.jpeg" height="240" width="320" /></a></div>
<br />
<br /></div>
<h3>
How Quickly Can You Get Data</h3>
<div>
<br />
The presentation that left me mesmerized was <a href="http://spark.apache.org/" target="_blank">Spark</a>! I can't wait to use it. It is a very compelling product and it's now backed up by <a href="http://www.cloudera.com/content/cloudera/en/home.html" target="_blank">Cloudera</a>. With Spark you can do the following:</div>
<div>
<ul>
<li>Get a compute engine for Hadoop data - no need to reinvent the wheel</li>
<li>Speed up! A 100% faster MapReduce engine</li>
<li>Sophisticated: it runs all the sophisticated algorithms. Get access to a library of sophisticated algorithms</li>
<li>A a big community behind it; the most popular Big Data open source (followed by Hadoop)</li>
<li>Learning from the big guys - Yahoo!, Conviva, and Cloudera are using it</li>
</ul>
<div>
Not to mention that it comes integrated with a analytic suite (Shark), a large-scale graph processing (Bagel), and real-time analysis (Spark Streaming). This is nice because rather than doing Hive, Hadoop, and Mahout, and Storm, I only have to learn one programming paradigm.<br />
<br /></div>
</div>
<h3>
How Fast Can You Show The Results</h3>
<div>
<br />
Twitter explains how they <a href="https://blog.twitter.com/2013/new-tweets-per-second-record-and-how" target="_blank">monitor millions</a> (+5,700 tweets per second) of Time Series. The presentation was superb. I found out that the stack that they're using, named "Observability", is composed on: <a href="http://twitter.github.io/finagle/" target="_blank">Finnagle</a>, <a href="http://cassandra.apache.org/" target="_blank">Cassandra</a>, and query language and execution engines based on <a href="http://www.scala-lang.org/" target="_blank">Scala</a>. Although is a work in progress the stack is about three years old. I hope that they open-sourced it stack so I can get more context on how they monitor a large distributed system. </div>
<div>
<br /></div>
<div>
Another very interesting product was Google's <a href="https://developers.google.com/bigquery/" target="_blank">Big Query</a>. This was one of those presentations in which we (my team and I) stumbled upon by accident. The presentation showed how to use Google's toolkit: <a href="https://developers.google.com/freebase/" target="_blank">Freebase</a>, <a href="https://developers.google.com/maps/" target="_blank">Maps</a>, and BigQuery to do analytics.<br />
<br /></div>
<h3>
It's All About Context, Results, or Charts</h3>
<div>
<br />
Another company that impressed me was <a href="http://www.trifacta.com/" target="_blank">Trifacta</a>. With their tool you can clean data, see the model (graph) and recursively do it again in case you see patterns or not. The tool is targeted to data scientists, data wranglers, and data analysts. It's a great tool to mine data data, but most important, you can clean the data and show the results with relative ease.</div>
<div>
<br /></div>
<div>
IPython: This rekindled my interest in Python. IPythons notebooks are great for data scientists. You can get code, text, and graphics all in one page, so it's the perfect tool to show quick results. It's not that Python wasn't a popular language for data scientists. NumPy library provides a solid MATLAB-like matrix data structure, with efficient matrix and vector operations. It also provides other great APIs like SciPy and Pandas.<br />
<br /></div>
<h3>
Big Data != Hadoop</h3>
<div>
<br />
Two topics that opened my eyes were <a href="http://mesos.apache.org/" target="_blank">Mesos</a> and <a href="http://hortonworks.com/hadoop/yarn/" target="_blank">YARN</a>. Mesos, what Twitter uses to manage its clusters, is similar to YARN (Yet Another Resource Negotiator). The Hadoop 2.0, or YARN, it's becoming more of an environment and operating system; not just a MapReduce. With YARN, the JobTracker is gone. The ResourceManager is what does the job of the JobTracker. The ResourceManager (RM) is a scheduler - it allocates resources based on a pluggable scheduling algorithm. RM manages and monitors all the applications, so it strictly limits to arbitrating available resources.</div>
<div>
<br /></div>
<div>
One of our favorite (me and two of my buddies), was <a href="http://strataconf.com/strata2014/public/schedule/detail/32271" target="_blank">Netflix Data Platform</a> by Kurt Brown. A different and a great presentation. Rather than going on the technology side, they explained how the culture is intertwined with their technology stack or decisions. For example, they talked about the reason for using "the cloud". Obvious reasons like: it's cheaper, much flexible (growth, a better place to do tests/spikes), and having multi data center is definitely a plus. Also, Amazon and RackSpace have great services such as SQL, EMR, and S3. But the main reason is "focus". They are focused on getting movies and increasing their audience rather than to focus on the "plumbing". They expressed their commitment to "open-source software" (OSS). They mentioned the great talent that they can get and how they can "manage their own destiny" by following these principles and using these tools.</div>
<div>
<br /></div>
<div>
Netflix explained their philosophy and how it's the "soul" of their decision (technical and business). For example, they keep keyboards, mice, and other peripherals in vending machines (they are free), so that everyone knows to "act in Netflix best interest". Furthermore, every decision or project needs to answer a basic question: "what value are you adding?". They apply the rule "accept that things will break". Because of this, they build safety nets around their systems. Again, it was a very nice and interesting presentation.</div>
<div>
<br /></div>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc2bDIhlg2ClnVyts4p-9p7U8B9jAWCKxj-s59dELTktJmzIIu8hczPb6uuz7ZKXn8JdYdPUtjr3oWkHfWgZcElLTM6nKTK-AkE-IM3446l7QAxSWLHF80W2VdUSLMwdk1ybADDraOWfg/s1600/strata3.jpeg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgc2bDIhlg2ClnVyts4p-9p7U8B9jAWCKxj-s59dELTktJmzIIu8hczPb6uuz7ZKXn8JdYdPUtjr3oWkHfWgZcElLTM6nKTK-AkE-IM3446l7QAxSWLHF80W2VdUSLMwdk1ybADDraOWfg/s1600/strata3.jpeg" height="240" width="320" /></a>I really enjoyed the conference. I also just purchased the <a href="http://shop.oreilly.com/product/110000131.do" target="_blank">videos</a>. Which I highly recommend!! During the next few months, I'm going to try to learn some of these tools and present them at the <a href="http://www.meetup.com/Miami-Java-User-Group/" target="_blank">Miami JVM Meetup</a>. Hopefully I can get to see you there, or better yet, hope to see you at Strata 2015. If you're going to either one of these events, let's meet up and share a beer...or two and discuss Big Data. I promise that my eyes will get dilated.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHxTSF_EjIPa1QqAWZMjbIqOzN6aY5CzwohxXdYbpMQZSmE5u6Di96PMNtw9fXHni8NAYOgiqIiDhtmCVCf14-FR7s3lqvo8XSTBjgG3cFQPgTmuktVePjaYVISyzuVelc2pXO7TKrTjg/s1600/strata2.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHxTSF_EjIPa1QqAWZMjbIqOzN6aY5CzwohxXdYbpMQZSmE5u6Di96PMNtw9fXHni8NAYOgiqIiDhtmCVCf14-FR7s3lqvo8XSTBjgG3cFQPgTmuktVePjaYVISyzuVelc2pXO7TKrTjg/s1600/strata2.jpeg" height="240" width="320" /></a></div>
<br />
<br /></div>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-27206951301424488712014-03-17T18:41:00.003-07:002014-03-21T08:10:24.401-07:00El Dilema de Ser Buen Samaritano o Come MierdaSiempre me a gustado ayudar a la gente, pero hay veces me pongo a pensar…soy "buen samaritano" o un "come mierda"? Desde pequeño me gusto salir y hablar. Mi mamá siempre me dijo que hablo “hasta por los codos”. Lo que he notado es que ahora, muchas personas me suelen hablar y pedirme por cosas. Algunas personas se acercan a mi para venderme algo, y otras para ayudarlas. La verdad es que muchas personas dicen que yo soy un muchacho “agradable”, otras personas dicen que soy “simpatico” (me gusta), yo a veces pienso que tengo cara de "come mierda”. Por ejemplo, es común que cuando voy a un mall, siempre las personas que tienen una tienda en un kiosco, siempre me llaman, “señor, le limpio el reloj?" o "Joven, tengo esto en especial.” Siempre termino en decir, “no gracias" con mi sonrisa y sigo adelante, para que…para encontrarme con otros dos muchacho/as que me van a preguntar lo mismo. Esto es muy común para mi. Mi esposa siempre me dice que vea abajo para que no me persigan, pero hasta eso! La otra ves, caminando en un estacionamiento con mi familia, una señora me paro en plena calle y me llamo. Luego me dijo, “me puedes hacer un favor, se me desamarro mi zapato, me lo amarras?” Y que es lo que hice? Pues lo amarre…como buen come mierda. Para consolarme, la señora era una anciana obesa. Pero aun así, de <i>todas</i> las personas, tuve que ser yo? Mi ultima escena de “buen samaritano” fue la otra ves que fui a desayunar con mi familia. Apenas salí de el carro, un señor me vio y me dijo que su carro lo encerro con las llaves dentro de el vehículo. Yo se, medio bruto el personaje, pero también yo soy super despistado - lo entiendo. “Me puedes llevar a casa para agarrar mis llaves de repuesto? Vivo bien cerca.” Mi esposa solo me vio, me dio un sonrisa, y me dijo que mientras iba a agarrar la mesa con los niños.<br />
<br />
La verdad muchas veces me pongo a pensar, “voy a ponerme así, todo cabrón y mandar al diablo a esa gente”. Pero no es como soy yo. Como dije, me encanta hablar! Cuando hice mi ultima “labor” de buen samaritano, le pregunte a el señor (algo mayor) que cuantas personas le había preguntado. El me respondió fui el primero, “tienes la pinta de ser amable”. Yo pensé, "mas bien, cara de come mierda.”<br />
<br />
Después de tanto tiempo así, viéndolo en retrospectiva, no solamente me da mucho agrado ayudar, pero también me ha ido muy bien con mis pequeñas labores. Como dije, se siente bien ayudar a la gente. Ademas, creo que da un buen ejemplo a mis dos hijos (tengo uno de doce y otra de 3 que cree que tiene trece). Y más aun cuando lo haces sin pensarlo mucho o pedir algo en cambio. Aunque nunca he pedido nada, siempre termino notando cosas bonitas. Como cuando el señor que dejo las llaves en el carro pago mi desayuno, en el caso de el año pasado alguien me compro un par de zapatos de $120, solamente porque tenia cara de buena gente. Al parecer, a los de cara de come mierda, tiene mucha suerte.<br />
<br />
Un abrazo!<br />
MarceloMarcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com9tag:blogger.com,1999:blog-9150005423109009329.post-36702334683236971412014-03-06T07:53:00.003-08:002014-03-06T07:53:50.457-08:00Disruptive Possibilities: How Big Data Changes Everything<div>
I was looking forward to this book because of the title. I was under the impression that I was going to find concrete examples on how Big Data has affected and disrupted some industries. Best of all, I thought that I was going to read what industries will be impacted and how. The book showed some examples at the end, but in my opinion, it leaves something very important: speed and sophistication. </div>
<div>
<br /></div>
<div>
I just came back from <a href="http://strataconf.com/strata2014" target="_blank">Strata 2014</a>, which is why I was looking forward to this book, and when I heard <a href="http://www.youtube.com/watch?v=KspReT2JjeE" target="_blank">Matei Zaharia's keynote</a>, it was all I needed to know about the current disruption of big data. Nowadays, big data storage is becoming commoditized, so the best value added is speed (how quick you can get the answer of your problem) and sophistication (run the best algorithms on the data). The book doesn't mentioned this but it might be because of its age - things are moving super quick on Big Data.</div>
<div>
<br /></div>
<div>
Some of the things that the book does well:</div>
<div>
<ul>
<li>Introduces some history about the Big Data problem</li>
<li>How it affected some of the silos technologies like RDBS</li>
<li>How they solve the scalability issue</li>
</ul>
</div>
<div>
If you are a manager or someone that has no understanding of the world of Big Data, then I would recommended. However, if you are a developer, data scientist, or data wrangler, then this book will be too basic. The one thing that I highly recommend, if you are interested in this subject, is to attend (or at least purchase the videos) of Strata.</div>
<div>
<br /></div>
<div>
You can get the book <a href="http://shop.oreilly.com/product/0636920029526.do" target="_blank">here</a>.</div>
<div>
<br /></div>
<div>
Happy reading,</div>
<div>
Marcelo</div>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-23050588387606132962013-12-26T11:42:00.001-08:002013-12-30T08:00:49.168-08:00My best decision of 2013<span style="font-family: Arial; font-size: 13px;">I started working at 3CInteractive (3Ci) as of March 29th, this is perhaps the best place that I’ve ever worked at. This is a brief post of why this was my best decision of the year. I didn’t want to write this post until the honeymoon period was over (usually after 4 months). There were three factors that helped me make the decision (and why I continue loving) to work at 3Ci: </span><br />
<ul>
<li><span style="font-family: Arial; font-size: 13px;"><b>Culture</b>: I wanted to work for a company who looked at IT as an innovation engine rather than a cost center.</span></li>
<li><span style="font-family: Arial; font-size: 13px;"><b>Team</b>: I wanted to work with awesome engineers (very motivated, smart, and with a lot of experience). Also, I wanted to be the dumbestperson in the room.</span></li>
<li><span style="font-family: Arial; font-size: 13px;"><b>Tackle big harry audacious goals</b>: To be part of a company whose products revolve around big problems that would (without a doubt) have an impact to the bottom line.</span></li>
</ul>
<span style="font-family: Arial; font-size: 13px;"></span><br />
<span style="font-family: Arial; font-size: 13px;">I was introduced to 3Ci in 2007 when I was trying to sell a startup that I was working for by the name of Up-Mobile. The startup wasn’t doing well in the US market, so I was handed the task of selling our customer based to another company. During that time, I met with 3Ci’s senior management and some of their architects. Their company was doing some similar things as our startup, unlike us though, they were able to evolve quick and found a potential market with top tier companies and a compelling product. Many of 3Ci programmers were very involved in open source projects. For good or bad, I was attached to Up-Mobile and I wasn’t ready to make a move. Nevertheless, 3Ci left a great impression. Later down the line, I became the organizer of the Miami Java User Group (MJUG) and Miami JVM Meetup. 3Ci was more than helpful to provide a venue, food, and beverage with no strings attached. There was a big turnaround of 3Ci employees when I did presentations. It turned out that they also do a lot of presentations on things that they are doing. Also, they are committed to open source projects while keeping away from the golden hammer anti pattern - if you have a hammer, everything looks like a nail. They had the motto of using the best tool for the problem.</span><br />
<div>
<span style="font-family: Arial; font-size: x-small;"><br /></span>
<br />
<h2>
<span style="font-family: Arial; font-size: 13px;">Culture</span></h2>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.3cinteractive.com/visuals/culture/events/3ci-events-DMwalk.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="213" src="http://www.3cinteractive.com/visuals/culture/events/3ci-events-DMwalk.png" width="320" /></a></div>
<span style="font-family: Arial; font-size: 13px;">When a lot of people think about the culture of a company, they think about the things that they can see and touch. When you enter 3Ci, you’ll see the graffiti on the walls, the cool paintings, the guitars, motorcycles, and the free food. But this is a byproduct of culture. At 3Ci we have three rules that are unbendable:</span><br />
<ul>
<li><span style="font-family: Arial; font-size: 13px;">Build a sustainable company in the emerging market for enterprise mobile services</span></li>
<li><span style="font-family: Arial; font-size: 13px;">Create a great culture that focuses on the personal and professional development of our team</span></li>
<li><span style="font-family: Arial; font-size: 13px;">Do important work for quality clients</span></li>
</ul>
<span style="font-family: Arial; font-size: 13px;"></span><br />
<span style="font-family: Arial; font-size: 13px;">I came from a couple of small companies or startups (with less than 20 people). When you work for a small company, you not only know everyone’s name, but you know what everyone is doing and why. Although 3Ci has more than 100 people, and many of them are distributed, everyone on the team understands what the goals are because of these three rules. The culture is what promotes the exceptional people working there, the proactive and ownership of tasks, breed result-focused, and the team. 3Ci's culture have created a set of values based on this culture:</span><br />
<span style="font-family: Arial; font-size: 13px;"></span><br />
<b><span style="font-family: Arial; font-size: 13px;">We are not in the business for the business alone but for a higher purpose - to make lives better, to solve important problems, and to enjoy what we do.</span></b>
<span style="font-family: Arial; font-size: 13px;">Our Data Scientist, Oliver, once said, “Don’t live a life to do a great work. Live a great life, and then the great work will follow”.</span><br />
<span style="font-family: Arial; font-size: 13px;"></span><br />
<span style="font-family: Arial; font-size: 13px;"><b>3Ci strives to be one of the best places you will ever work.</b></span><br />
<span style="font-family: Arial; font-size: 13px;">Time and time again, you see that we become either the best place in South Florida to work or America’s most promising company.</span><br />
<span style="font-family: Arial; font-size: 13px;"></span><br />
<span style="font-family: Arial; font-size: 13px;"><b>We’re in it together</b></span><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCxKyzPf0wQXj0uzQG2kz5MU3Q4BmvlBUXL6GZlrBt1V600VBTDUWfC3WPLPG6CCN_-MU54THmGcVJOR6uYZAWTm2R6EHfdSBawdjGezaruj2KuypFjT4mzL17_TCTDoVllDO9cZ5xH-8/s1600/2013-10-21+10.20.51.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCxKyzPf0wQXj0uzQG2kz5MU3Q4BmvlBUXL6GZlrBt1V600VBTDUWfC3WPLPG6CCN_-MU54THmGcVJOR6uYZAWTm2R6EHfdSBawdjGezaruj2KuypFjT4mzL17_TCTDoVllDO9cZ5xH-8/s320/2013-10-21+10.20.51.jpg" width="320" /></a></div>
<span style="font-family: Arial; font-size: 13px;">There is a sense of, “let me know if I can help” within 3Ci. It’s also okay to fail as long as you learn from your mistakes. The head of our data team, Gabe, once said to me, “Do not do things when you are frustrated or under pressure. Rather do them when you're calm. You don’t want to make things worse by doing yet another mistake”. He is also the first one that holds my feet to the fire when something goes wrong. The same goes with Alex, my boss and head of the Software Engineering. It’s not a “don’t let it happen again”. It’s more like, “We really messed up. Why did it happen? What’s the root-cause of the problem. Can we make sure that this doesn’t happen again through some process? Can we automate this issue?"</span><br />
<span style="font-family: Arial; font-size: 13px;"></span><br />
<span style="font-family: Arial; font-size: 13px;"><b>We’re champions of change</b></span><br />
<span style="font-family: Arial; font-size: 13px;">We need to change quickly and adapt. Anyone with some experience knows this, but the one that are constantly thinking are those entrepreneurs that work at startups. I learned early on my career that the life a technology company is composed of three: you either go big, stay small, or be eaten. This was mentioned by Mike FitzGibbon (Fitz), president and cofounder, at the company's "All Hands Call", we need to be customer-centric and be able to quickly and effectively execute goals.</span><br />
<div>
<span style="font-family: Arial; font-size: x-small;"><br /></span>
<br />
<h2>
<span style="font-family: Arial; font-size: 13px;">Team</span></h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmBtWgPEmCE4FEEKNQvOF2iOuFSQ5aDZF0K_Di7jYDlajvA1NzKJNqs294XL1Us4nBlkgY8pjUNbD3FXhyRRS0talU92qDg1Hfq0GBLrapu3yRTup3TWWN1-y8RGNpvEL5FMbQtSwrXBI/s1600/2013-10-25+montevideo_0387.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmBtWgPEmCE4FEEKNQvOF2iOuFSQ5aDZF0K_Di7jYDlajvA1NzKJNqs294XL1Us4nBlkgY8pjUNbD3FXhyRRS0talU92qDg1Hfq0GBLrapu3yRTup3TWWN1-y8RGNpvEL5FMbQtSwrXBI/s400/2013-10-25+montevideo_0387.jpg" width="400" /></a></div>
<span style="font-family: Arial; font-size: 13px;">One of the main engineers at 3Ci during 2007 was Alejandro (Alex), and I met him on an open source project (Kannel). I checked some of his code, and I really like what he was doing. But the best things about him was his curiosity. He was a system administrator at 3Ci, yet he was learning Java at that time and committing to Kannel. I was also a good friend with the one of the main committers and founder of Kannel, Stipe Tolj. Stipe was also working for 3Ci as a consultant and knew Alex personally. He told me about some of the things that the company was doing, and I was intrigued to say the least. After a couple of e-mails, Alex and I pretty much hit it off. At that time he was working in Europe, then he was promoted and moved to 3Ci (Boca Raton, FL). The moment that I found out, I immediately sent him my resume. During my interview, I met a couple of guys: Mauricio and Carlos (Carlitos). Two of their best architects. This is another way of knowing how good the company, they had a good technical interview process. The interview was hard but I did well, so I was hired. I’ve never looked back. I highly enjoy it, and it’s the reason of why I make the 1.5 hour commute on Tuesday and Thursday from Kendall to Boca Raton.</span><br />
<span style="font-family: Arial; font-size: 13px;"><br /></span>
<span style="font-family: Arial; font-size: 13px;">One of my favorite quotes is "Hire the most amazing people you can. Communicate goals. Turn them loose. Profit." - Sam Schillance, Box. One of my rules of working at a job is to be surrounded by great engineers, then see how much they complain about my code. This is by far why I love this company. Again, It’s not the graffiti, nor the free food (which is delicious), nor the fact that I work remotely from Miami. It’s the engineers that I work with. When I started, I met a few great engineers. People from my team are former Googlers, people who worked at Bloomberg, current Apache committers, and entrepreneurs. Just a great set of interesting people who love to code and yet find a way to find a life after work - teachers, band players, great parents, rock climbers, etc. For the past nine months, I’ve learn more than any other company in my professional career, both personally and technically. Don’t get me wrong, it’s a very competitive company! The bar is raised high the moment you are hired, and the expectations are tremendous, but the safety net of the team is why I have a smile on my face every time I come to work. For example, there are five people that I always turn to in case I have any questions: Gabe, Alex, Tyler, Oliver, and Rob. In my view, these guys composed the core of the technology of 3Ci. They have a tons of experience, are highly innovated, smart, charismatic, and a great set of guys. Together, I call them my G.A.T.O.R team and it’s such a pleasure working with them.</span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial; font-size: 13px;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAlFd_SaJOjEktDWF6L3uEIqoCpN8E7u5EB9oZbWAfwINjlCDm9dH1XpHg1hVG-DkSqo3KYI9muRdiyJETgPf5v1cL9rjeovLHUizhKqbGaI75ZMuhnS_iQrugWsz4BkOHwa8yYsIRpYs/s1600/2013-10-23+20.40.25.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhAlFd_SaJOjEktDWF6L3uEIqoCpN8E7u5EB9oZbWAfwINjlCDm9dH1XpHg1hVG-DkSqo3KYI9muRdiyJETgPf5v1cL9rjeovLHUizhKqbGaI75ZMuhnS_iQrugWsz4BkOHwa8yYsIRpYs/s400/2013-10-23+20.40.25.jpg" width="400" /></a></span></div>
<br />
<span style="font-family: Arial; font-size: 13px;"><br /></span>
<span style="font-family: Arial; font-size: 13px;">My final interview was with one of the founders and COO, Mark Smith. One of his questions was, “What do you want to do in the future?”. I told him that I wanted to eventually start my own company, learn Machine Learning, and keep running MJUG and Miami JVM Meetup. He said, “Awesome, we’ll help you”. Done! After that, I was asking “Where do I sign?" Mark and Alex are by far the best leaders that I ever had. They know when to help the team, and when to get out of their way. If you worked for more than 10 years, then you’ll know that it’s hard to find a good company, even harder to find a good leader that could be a mentor.</span><br />
<div>
<span style="font-family: Arial; font-size: x-small;"><br /></span>
<br />
<h2>
<span style="font-family: Arial; font-size: 13px;">Big Hairy Audacious Goals</span></h2>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzawfTtJrOYVC8ljVlfphoz9NnPEdhPBL7OQdFLAuWbiFnHT1ds5qkXcugk1F45pQrI8OISvwHRdhET4sQ7QPfteAOqBTFgUJGzopp5AF5nwSiNpCKd37gB0A0H2f0MfksM32LavIJw0A/s1600/math-problem.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzawfTtJrOYVC8ljVlfphoz9NnPEdhPBL7OQdFLAuWbiFnHT1ds5qkXcugk1F45pQrI8OISvwHRdhET4sQ7QPfteAOqBTFgUJGzopp5AF5nwSiNpCKd37gB0A0H2f0MfksM32LavIJw0A/s320/math-problem.jpg" width="320" /></a></div>
<span style="font-family: Arial; font-size: 13px;"><br /></span>
<span style="font-family: Arial; font-size: 13px;">Another huge incentive for me was to be able to tackle what James Collins calls, “Big Hairy Audacious Goals”. Although I can’t say anything about the goals for the company, trust me, they are as big as they are interesting. This is what motivates me. The fact that I work with an awesome team tackling some really big problems, help set the path of pushing the envelop as much as we can. Because of this, we look at other technologies and think outside the norm. I believe that there is a challenge-to-great-developers matrix </span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial; font-size: 13px;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOqtSoegox1YySJXb-QB-v3Qqo6b5_msrPc9GeGfV8LrXFNLTxr5sJpIgrKFoN6fDkzUkFmEJqXaZv22VHy3HKgB9gG7GO_VW2rpGHwh-6lFnhPHxi8zBx0ysoVUjQDkTtecbSvIs61e8/s1600/matrix.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOqtSoegox1YySJXb-QB-v3Qqo6b5_msrPc9GeGfV8LrXFNLTxr5sJpIgrKFoN6fDkzUkFmEJqXaZv22VHy3HKgB9gG7GO_VW2rpGHwh-6lFnhPHxi8zBx0ysoVUjQDkTtecbSvIs61e8/s320/matrix.jpg" width="320" /></a></span></div>
<span style="font-family: Arial; font-size: 13px;">. In my experience, great engineers gravitate to hard challenges. This is where I want to be. I don’t want to work on another CRUD application (been there, done that, got the t-shirt!) I believe that the best combination of retention of great engineers is to have great challenges for them to solve, have a great culture, and provide the best atmosphere for them.</span><br />
<span style="font-family: Arial; font-size: 13px;"></span><br />
<span style="font-family: Arial; font-size: 13px;">I’ve always want to work for a company like 3Ci and I’m glad to be part of it. It’s hard to find companies such as these in South Florida. 3Ci is committed to hire the best. They are also very much focused on the culture. The best way that I can describe its culture is using this quote, “When you combine a culture of discipline with an ethic of entrepreneurship, you get the magical alchemy of great performance" - Jim Collins, Good to Great. I once read that you should choose a profession that you enjoy and that serves as many people as possible. Focusing on serving others - not on building wealth. Serve well and money will follow. I’m glad that this year I was able to scratch that one from my “to-do” list.</span><br />
<span style="font-family: Arial; font-size: 13px;"></span><br /></div>
</div>
</div>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com1tag:blogger.com,1999:blog-9150005423109009329.post-35176169633369400502013-06-04T15:57:00.001-07:002013-06-04T16:16:29.507-07:00Functional Thinking Video ReviewI view new programming languages like president candidates, I don't trust them. They believe that by being in office (projects), everything will be better. When functional programming started its hype I watched from afar. This was until I was working for a stock trading firm with lots of financial algorithms and lots of multithreaded application. Before this gig, my languages of choice have been pure Java and Groovy. The world of Scala, Clojure, Haskell, and Erlang was just a bunch of noise. I was skeptical about this presentation, but I am a fan of <a href="http://nealford.com/" target="_blank">Neal Ford</a> so I decided to give it a shot. Overall, I was very pleased with the content mainly because it did not focused on syntax, it focused on context! "Functional" is more a way of thinking than a tool set. For anyone to understand functional programming you need to understand the concepts, and Neal achieved this in his presentation.<br />
<br />
Neal points out the major advantages of using functional programming:<br />
<br />
<ol>
<li>Language Evolution: all major languages are adding functional features. </li>
<li>Results over steps: create optimized applications to solve a problem rather than using frameworks</li>
<li>Immutability: the freedom of not worrying about the state of the objects - "failure atomicity" </li>
</ol>
Then, he elaborates a bit more on the subject matter. For example:<br />
<br />
<ul>
<li>First-class/higher-order functions </li>
<li>Pure Functions </li>
<li>Recursion </li>
<li>Strict Evaluation </li>
</ul>
If you want to see more, check out the videos <a href="http://shop.oreilly.com/product/0636920030393.do">here</a> or check <a href="http://www.youtube.com/watch?feature=player_embedded&v=plSZIkLodDM">this</a> video for introduction.Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-18480933997515586592013-05-31T10:33:00.002-07:002013-05-31T15:10:10.391-07:00Configure MTR in MacTo configure MTR do the following:<br />
First Install <a href="http://mxcl.github.io/homebrew/">brew</a><br />
Install MTR using the following commands:<br />
<script class="brush:java" type="syntaxhighlighter">
brew install mtr --no-gtk<br />
</script>
Unfortunately, the configuration installs in the /usr/local/sbin<br />
<script class="brush:java" type="syntaxhighlighter">
ll /usr/local/bin/mtr<br />
lrwxr-xr-x 1 root admin 35 May 31 13:11 /usr/local/sbin/mtr -> /usr/local/Cellar/mtr/0.82/sbin/mtr<br />
</script>
What I end up doing is removing the symbolic link and point it to my bin<br />
<script class="brush:java" type="syntaxhighlighter">
rm /usr/local/sbin/mtr<br />
ln -s /usr/local/Cellar/mtr/0.82/sbin/mtr /usr/local/bin/mtr<br />
lrwxr-xr-x 1 root admin 35 May 31 13:11 /usr/local/bin/mtr -> /usr/local/Cellar/mtr/0.82/sbin/mtr<br />
#Now you should be able to run it using sudo
sudo mtr google.com
My traceroute [v0.82]
mbp152XF1G3.local (0.0.0.0) Fri May 31 13:33:15 2013
Resolver: Received error response 2. (server failure)er of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. 127.0.0.1 0.0% 6 1.1 1.2 1.0 1.6 0.2
2. 96.124.35.101 0.0% 6 7.8 9.5 7.8 15.8 3.1
3. te-9-3-ur01.somewhere.us.c 0.0% 6 10.2 9.3 8.5 10.2 0.7
4. te-0-4-0-8-ar03.somewhere.us.pom 0.0% 6 12.6 14.1 11.7 17.9 2.6
5. he-2-7-0-0-cr01.somewhere.usa.c 0.0% 5 23.4 16.9 12.0 23.4 4.3
6. be-10-pe02.somewhere.us.comcast 0.0% 5 11.8 11.7 11.3 12.0 0.3
7. 96.30.206.118 0.0% 5 10.3 10.9 10.0 12.4 0.9
8. 222.23.253.74 0.0% 5 10.0 10.9 10.0 11.6 0.8
9. 212.85.241.87 0.0% 5 15.2 12.2 10.7 15.2 1.7
10. ddd05s17-in-f7.1e100.net 0.0% 5 10.1 10.3 9.8 10.8 0.4
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-18625634933827892552013-03-07T09:20:00.001-08:002013-03-07T09:22:04.769-08:00Personal MBA Book ReviewJust finished reading <a href="http://www.amazon.com/Personal-MBA-Master-Art-Business/dp/1591845572" target="_blank">The Personal MBA</a>: Master the art of business by Jorge Kaufman. I have an Executive MBA from the University of Miami (UM or "The U"), so I was very skeptical at the beginning. However, I really enjoyed the book. First of all, there is a lot of stuff that I learned during my MBA, but as he puts it towards the end of the book, <br />
<blockquote>
Educating yourself about anything is a Tao - there's no end to the process. The journey itself is the reward.</blockquote>
If you are the type of person that wants to learn more about business, I highly recommend it. The book <a href="http://personalmba.com/best-business-books/" target="_blank">distilles a series amount of great books</a> into one. Here's what you learn:<br />
<ul>
<li>how businesses work</li>
<li>how people work (chapter 6 -8)</li>
<li>how systems work (chapter 9-11) </li>
</ul>
You shouldn't expect:<br />
<ul>
<li>Manager and leadership overload. </li>
<li>Finance and accounting. </li>
<li>Financial intelligence for entrepreneurs </li>
<li>How to read a financial report </li>
<li>Quantitative analysis: thinking statistics, turning numbers into knowledge </li>
</ul>
The one thing that I kept using while reading was a notebook and pen handy. I also tried to review this every month.<br />
<br />
I found it kind of sad that the book didn't covered any quantitative analysis in this analytic age, but I was happy that he covered how systems work. This is the chapter that really resonated with me. Here are my notes on the systems chapters.<br />
<br />
<h1>
Analyzing Systems</h1>
<b>Deconstruction</b>: the only way to analyze a complex system is by deconstructing it. "Deconstruction is the process of separating complex systems into the smallest possible sybsystems in order to understand how things work." You need to find the "ins" and "outs" of the system for you to understand it. Unable to understand complex systems, you need to decompose them into smaller systems for you to understand them.
<br />
<br />
<b>Measurement</b>: You need to measure the systems so you know if it's doing fine or not. "Measurement helps us avoid the absence blindness when analyzing systems. Remember: we have a hard time seeing things that aren't present. Measuring different parts of a system in operation helps to identify potential issues before they arise." One example is diabetes, a person needs to measure its blood glucose to know if it's too high or too low. A person cannot tell the level of glucose unless he/she measures it.
<br />
<br />
<b>Key Performance Indicator (KPI)</b>: some measurements are better than others. Key performance indicator tell you exactly the status of your system. "Key performance indicators are measurements of the ciritical parts of a system. Measurements that don't help you make improvements to your system are worse than worthless: they are a waste of your limited attention and energy." Few questions the author used to identify business' KPI:
<br />
<br />
<ul>
<li>Value Creation: how quickly is the system creating value? What is the current level of inflows?</li>
<li>Marketing: how many people are paying attention to your offer? How many prospects are giving you permission to provide more information?</li>
<li>Sales: how many prospects are becoming paying customers? What is the average customer's lifetime value?</li>
<li>Value Delivery: how quickly can you server each customer? What is your current returns or complaints rate?</li>
<li>Finance: what is your profit margin? How much purchasing power do you have? Are you financially sufficient?</li>
</ul>
<br />
<b>Analytical Honesty</b>: always look into data without discrimination of personal feeling. "Analytical honesty means measuring and analyzing the data you have dispassionately. Since humans are social creatures, we tend to care deeply about how other perceive us, which give us a natural incentive to make things look better than they actually are. If you purpose is to actually make things better, this tendency can get in the way of collecting accurate data and conducting useful analysis.<br />
<div>
<br />
<b>Sampling</b>: is the process of taking at random a small percentage of the total output, then using it as a proxy for the entire system. If you ever had blood taken at the doctor's office, you'll have a good idead of what sampling entails. </div>
<div>
<br />
<b>Margin of Error</b>: this entails on the percentage of accuracy your system has based on tests. "Is an estimate of how much you can trust your conclusions from a given set of observed samples." When it comes to analytical confidence, more data is always better. </div>
<div>
<br />
<b>Ratio</b>: is a method of comparing two measurements against each other. By diving your results by your input, you can measure all sorts of useful relationships between different parts of your system. Here are some useful ratios to track:
<br />
<ul>
<li>Return on promotions: for every $1 you spend in advertising, how much revenue do you collect?</li>
<li>Profit from Employee: for every person you employ, how much profit does your business generate?</li>
<li>Closing Ratio: for every prospect you serve, how many purchase?</li>
<li>Returns/Complains Ratio: for every sale you make, how many choose to return or complain?</li>
</ul>
<b>Typically</b>: identifying a normal or typical value for some important measurement. There are four common methods of calculating a typical value: mean, median, mode, and midrange.
<br />
<ul>
<li>mean: average</li>
<li>median: sort the values in order of high to low, then finding the quantity of the data point in the middle of the range. Median are actually a specific form of analysis called a percentile: the median is the value that expresses the fiftieth percentile. By definition, 50% of the values in the set will be below the median.</li>
<li>mode: is the value that occurs most frequently in a set of data. Modes are useful for finding clusters of data
- a set can have multiple modes, which can alert you to potentially interesting interdependencies in the system that produced that data.</li>
<li>midrange: is the value halfway between the highest and lowest data points i a set of values. To calculate the midrange, add the highest and lowest values, then divide by two. Midranges are best used for quick estimates - they're fast, and you only need to know two data points, but they can be easily skewed by outliers that are abnormally high or low - Bill Gate's bank balance.</li>
</ul>
<br />
<b>Correlation and Causation</b>: causation is a complete chain of cause and effect. Correlation is not causation. Just because you can here illegal doesn't make you a criminal. Causation is always more difficult to prove than correlation. When analyzing complex systems with many variables and interdependencies, it's often extremely difficult to find true causality. </div>
<div>
<br />
<b>Norms</b>: are measures that use historical data as a tool to provide context for current measurements. If you are selling Christmas ornaments, it's a waste to compare them Q4 with Q2. You should compare this Q4 with the previous Q4. </div>
<div>
<br />
<b>Proxy</b>: measures one quantity by measuring something else. Used with care, proxies can help you measure the measurable - just be sure your proxy is directly and highly correlated with the subject of interest.
<br />
<br />
<b>Segmentation</b>: is a technique that involves splitting a dataset into well-defined subgroups to add additional context. Finding out that last quarter's sales increased by 20% is good, but knowing that they were done by 80% of women is even better. There are three common ways to segment customer data: past performance, demographics, and psychographics.
<br />
<ul>
<li>Past performance: segments customers by past known action. For example, you can segment customer sales data using previous sales data, comparing sales to new customers with sales to customers who have previously purchase from you. Lifetime value calculations are a form of segmentation by past performance.</li>
<li>demographics segments customers by external personal characteristics. Personal information like age, gender, income, nationality, and location can help you determine which customers are your probable purchasers.</li>
<li>psychographics: segments customers by internal psychological characteristics. Typically discovered via surveys, assessments, or focus groups.</li>
</ul>
<br /><ul>
</ul>
<h1>
Improving Systems:</h1>
<b>Optimization</b>: this is the process of maximizing the output of a system or minimizing a specific input the system requires to operate. Optimization typically revolves around the systems and processed behind your "key performance indicators", which measure the critical elements of the system as a whole. Improve your KPIs, and your system will perform better. </div>
<div>
<br />
<b>Refactoring</b>: the process of re-engineering the systems to improve efficiency without changing its output. The primary benefit of refactoring isn't improving the output - it's making the system itself faster or more efficient. </div>
<div>
<br />
<b>The Critical Few</b>: this is the 80/20 rule from Vilfredo Pareto. <br />
<blockquote>
For best results, focus on the critical inputs that produce most of the results you want.</blockquote>
Finding the inputs that produce the outputs you want, then make them the focus of your time and energy. Ruthlessly weed out the rest. </div>
<div>
<br />
<b>
Diminishing Return</b>: after a certain point, having more of something can actually be detrimental. In the book the author explains about his work at P&G. At this company, he was the one in charge of analyzing the results of his advertising (tv spots). After time, the commercial will "wear out". That's the effect of diminishing return. "Optimize and refacgtor up to the point you start experiencing diminishing returns, then focus on doing something else".
<br />
<br />
<b>Friction</b>: Every business process has some amount of friction. The key is to identify areas where friction currently exists, then experiment with small improvements that will reduce the amount of friction in the system.
<br />
<br />
<b>Automation</b>: refers to a system or process that can operate without human intervention. <br />
<blockquote>
Find a way to automate your system, and you open the doors to scale via duplication and multiplication, improving your ability to create and deliver value to more paying customers.</blockquote>
<br />
<b>The Paradox of Automation</b>: the more efficient the automated system, the more crucial the contribution of the human operators of that system. When an error happens, operators need to identify and fix the situation quickly or shut the system down - otherwise, the automated system will continue multiplying the error. </div>
<div>
<br />
<b>The Irony of Automation</b>: the more reliable the system, the less human operators have to do, so the less they pay attention to the system while it's in operation. Remember the Mackworth Clock and vigilance studies conducted on British radar operators in WWII from our discussion on Novelty? Humans get bore extremely quickly if things stay the same, and the more reliable the system, the more things stay the same. </div>
<div>
<br />
Keep your automated system's operators mentally engaged, and they'll be far more likely to notice when errors inevitably occur. </div>
<div>
<br />
<b>Standard Operating Procedure (SOP)</b>: is a predefined process used to complete a task or resolve a common issue. Business systems often include repetitive tasks or resolve a common issue. Well-defined standard operating procedures are useful because they reduce friction and minimize will power depletion. Instead of wasting valuable time and energy solving a problem that has already been solved many times before, a predefined SOP ensures that you spend less time thrashing and more time adding value. </div>
<div>
<br />
Don't let your standard operating procedure lapse into bureaucracy. Remember, the purpose of an SOP is to minimize the amount of time and effort it takes to complete a task or solve a problem effectively. If the SOP requires effort without providing value, it's friction. </div>
<div>
<br />
<b>Checklist</b>: a checklist is an externalized, predifined standard operating procedure for completing a specific task. Checklist will help you define a system for a process that hasn't yet been formalized. Second, using checklists as a normal part of working can help ensure that you don't forget to handle important steps that are easily overlooked when things get busy. </div>
<div>
<br />
Checklist can produce major improvements in your ability to do quality work, as well as your ability to delegate work effectively. By taking the time to explicitly describe and track your progress, you reduce the likelihood of major errors and oversights, as well as prevent willpower depletion associated with figuring out how to complete the same task over and over again. </div>
<div>
<br />
<b>Cessation</b>: is the choice to intentionally stop doing something that's counterproductive. In The One-Straw Revolution, Masanobu Fukuoka wrote about his experiments with natural farming, which mostly involved letting nature take its course and intervening as little as possible. Instead of trying to do too much, Fukuoka only did what was absolutely necessary. As a result, his fields were consistently among the most productive in the are. </div>
<div>
<br />
Cessation takes guts. It's often unpopular to unpalatable to do nothing even if doing nothing is actually the right solution. </div>
<div>
<br />
<b>Resilience</b>: what business needs is more turtles and fewer tigers. Turtles are nature's tank - they can eat different food, go through hibernation if times are tough, and shelter from an enemy if needed. Tigers depends on their strength, speed, and pray to live. If prey becomes scarce or they lose their hunting powers due to age or injury, death takes them quickly and mercilessly. This is why turtles live longer than tigers. </div>
<div>
<br />
Resilience is a massively underrated quality in business. Having the toughness and flexibility to handle anything life throws at you is a major asset that can save your skin.
<br />
What makes a business resilient:
<br />
<ul>
<li>low (preferably zero) outstanding debt</li>
<li>low overhead, fixed costs, and operating expenses</li>
<li>substantial cash reserves for unexpected contingencies</li>
<li>multiple independent product/industries/lines of business</li>
<li>flexible workers/employees who can handle many responsibilities well</li>
<li>
</li>
<li>no single points of failure</li>
<li>fail-safes/backup systems for all core process</li>
</ul>
<blockquote>
Planning for resilience as well as performance is the hallmark of good management.</blockquote>
<b>Fail-safe</b>: is a backup system design to prevent or allow recovery from a primary system failure. As much as possible never have a single critical point of failure. </div>
<div>
<br />
<b>Stress testing</b>: is the process of identifying the boundaries of a system by simulating specific environmental conditions. </div>
<div>
<br />
<b>Scenario Planning</b>: is the process of systematically constructing a series of hypothetical situations, then mentally simulating what you would do if they occurred. Most large business use scenario planning as the basis of a practice called "hedging": purchasing various forms of insurance to reduce the risk of unfavorable future events. </div>
<div>
<br />
Don't waste time trying to predict an unknowable future - construct the most likely scenarios and plan what you'll do if they occur, and you'll be prepared for whatever actually happens. </div>
<div>
<br />
<b>The Middle Path</b>: is the ever-changing balance point between too little and too much - just enough. </div>
<div>
<br />
<b>The Experimental Mind-set</b>: constant experimentation is the only way you can identify what will actually produce the result you desire. Often, the best (or only) way to learn things is to jump in and try.
<br />
You learn the most from what doesn't go well. As long as your mistakes don't kill you, paying attention to what doesn't work can give you useful information you can use to discover what does.
<br />
<br />
<br /></div>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-72807495395925080322013-02-28T07:28:00.001-08:002013-02-28T07:29:30.906-08:00Using Deadbolt for Authorization in Your Play 2 Site With Scala<p>
I needed security in my site. I needed a feature that would guarantee for only authorized users to go into some of the controllers (links) of my site. For non-registered users, I wanted to display a "non authorized" page so they could either login or register. I also needed for Play 2.1 using Scala 2.10. My search lead me to <a href="https://github.com/schaloner/deadbolt-2-guide/blob/master/03-root-concepts.markdown">deadbolt 2 (D2)</a>. In this post I will cover the following:
<ul>
<li>How to include deadbolt into your Play site</li>
<li>Create a security service to validate registered users</li>
<li>Send non-registered users to a "not authorized" page</li>
<li>Add security to your controller</li>
<li>How to validate registered users from the template</li>
</ul>
</p>
<h2>Include deadbolt into your Play site</h2>
<p>The first thing that you need is to include the deadbolt plugin into your site. You need to include it inside your project/Build.scala. Here you will find that I added the play-plugin-mailer already, postgresql, and finally deadbolt. I had to include the resolvers for the application to work.</p>
<script class="brush:java" type="syntaxhighlighter">
import sbt._
import Keys._
import PlayProject._
object ApplicationBuild extends Build {
val appName = "acme"
val appVersion = "1.0-SNAPSHOT"
val appDependencies = Seq(
jdbc,
anorm,
"com.typesafe" %% "play-plugins-mailer" % "2.1.0",
"postgresql" % "postgresql" % "9.1-901-1.jdbc4",
"be.objectify" %% "deadbolt-scala" % "2.1-SNAPSHOT"
)
val main = play.Project(appName, appVersion, appDependencies).settings(
// Change this to point to your local play repository
resolvers += Resolver.url("Objectify Play Repository", url("http://schaloner.github.com/releases/"))(Resolver.ivyStylePatterns),
resolvers += Resolver.url("Objectify Play Repository - snapshots", url("http://schaloner.github.com/snapshots/"))(Resolver.ivyStylePatterns)
)
}
</script>
<h2>Add Security Service to Your Application</h2>
<p>
Before you implement the service, you need to create a subject. Subject is what determines a user in deadbolt. It also handles privileges such as roles and permissions. Below is the code for a User inside the app/models package:
</p>
<script class="brush:java" type="syntaxhighlighter">
package models
import play.libs.Scala
import be.objectify.deadbolt.core.models.Subject
class User(val userName: String) extends Subject {
def getRoles: java.util.List[SecurityRole] = {
Scala.asJava(List(new SecurityRole("foo"),
new SecurityRole("bar")))
}
def getPermissions: java.util.List[UserPermission] = {
Scala.asJava(List(new UserPermission("printers.edit")))
}
def getIdentifier: String = userName
}
</script>
<p>Now, that we have a subject, we need to implement the DeadboltHandler to validate registered users/subjects and to send the non-register users to a "non-register" page. The main application that takes care of user validation is the getSubject.
</p>
<script class="brush:java" type="syntaxhighlighter">
override def getSubject[A](request: Request[A]): Option[Subject] = {
val session = username(request)
session match {
case Some(session) => Some(new User(username(request).get))
case None => None
}
}
</script>
<p>
Here the code tries to get the the username from the HTTP session, and in case there is a user, return a subject (User) with that session name. Otherwise, return a "None". When returning None, Deadbolt automatically knows that this is not a valid/registered user.
</p>
<h2>Send Non-Registerd users to a "Non-Registered" Page</h2>
<p>For those users that are not registered (None Subject), the "onAuthFailure" will take effect.</p>
<script class="brush:java" type="syntaxhighlighter">
def onAuthFailure[A](request: Request[A]): Result = {
Logger.debug("authentication failure")
Results.Forbidden(views.html.accessFailed())
}
</script>
<p>Below is the code inside the app/security package.</p>
<script class="brush:java" type="syntaxhighlighter">
package security
import be.objectify.deadbolt.scala.{ DynamicResourceHandler, DeadboltHandler }
import play.api.mvc.{ Request, Result, Results }
import be.objectify.deadbolt.core.models.Subject
import play.api.mvc._
import models.User
import play.Logger
class MyDeadboltHandler(dynamicResourceHandler: Option[DynamicResourceHandler] = None) extends DeadboltHandler {
def beforeAuthCheck[A](request: Request[A]) = None
override def getDynamicResourceHandler[A](request: Request[A]): Option[DynamicResourceHandler] = {
if (dynamicResourceHandler.isDefined) dynamicResourceHandler
else Some(new MyDynamicResourceHandler())
}
/**
* Retrieve the connected user email.
*/
private def username(request: RequestHeader) = request.session.get(Security.username)
override def getSubject[A](request: Request[A]): Option[Subject] = {
val session = username(request)
session match {
case Some(session) => Some(new User(username(request).get))
case None => None
}
}
def onAuthFailure[A](request: Request[A]): Result = {
Logger.debug("authentication failure")
Results.Forbidden(views.html.accessFailed())
}
}
</script>
<h2>Add Security to the Controller</h2>
<p>
Now, that the DeadboltHandler is implemented, you have to make some decisions based on the business rules of your site. For example, if you have an Contact Directory page where only registered users could see it, you need to assure that there is a Subject (registered user) present.
</p>
<script class="brush:java" type="syntaxhighlighter">
package controllers
import play.api._
import play.api.mvc._
import play.api.i18n.Messages
import models.User
import security.{MyDeadboltHandler}
import be.objectify.deadbolt.scala.DeadboltActions
object ContactDirectory extends Controller with DeadboltActions {
def contacts = SubjectPresent(new MyDeadboltHandler) {
Action { implicit request =>
Ok(views.html.contacts(new MyDeadboltHandler))
}
}
}
</script>
<p>
Noticed that I am also passing a new MyDeadboltHandler to the page. This is because I also need to implement some business rules inside the template. In this site, I needed to show a "Login" link for those non-registered users or display a "Contact Directory" and a "Logout" link for those authenticated users. Below is my navigation template.
</p>
<script class="brush:java" type="syntaxhighlighter">
@(handler: be.objectify.deadbolt.scala.DeadboltHandler)(implicit request: Request[Any])
<div class="container">
<div class="navbar">
<div class="navbar-inner">
<div class="container">
<div class="nav-collapse">
<ul class="nav nav-pills">
<li class="@if(page.equals("HOME")) {active }single"><a href="@routes.Application.home">HOME </a></li>
@subjectNotPresent(handler) {
@notlogin(handler)
}
@subjectPresent(handler) {
@loggedin(handler)
}
</ul>
</div>
</div>
</div>
</div>
</div>
</script>
<p>Inside the "subjectNotPresent" we will handle the case when the user is not logged in. Whereas, in the subjectPresent we will show the Logout and the Contact Directory link pages.
</p>
<p>For those controllers that doesn't require the a subject to be present, I still had to pass the implemented deadbolt handler because it had to be in every one of my pages. This is to validate if the user is authenticated.
</p>
<script class="brush:java" type="syntaxhighlighter">
package controllers
import play.api._
import play.api.mvc._
import security.MyDeadboltHandler
object Application extends Controller {
def home = Action { implicit request =>
Ok(views.html.index(new MyDeadboltHandler))
}
}
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com3tag:blogger.com,1999:blog-9150005423109009329.post-23485033013478550992013-02-22T13:03:00.001-08:002013-02-22T13:05:58.197-08:00Non-Blocking Async with Scala 2.10 and Play 2.1<p>
In my application, I needed to consume an "expensive" (+3 seconds to respond) web service. The results had to be as fast as possible without halting the system/process. What I needed, was to process the web service in the background (thread), then display the results in the application. I used Twitter and GitHub as an example.
</p>
<script class="brush:java" type="syntaxhighlighter">
package controllers
import play.api._
import play.api.mvc._
import play.api.libs.ws.WS
import play.api.libs.concurrent.Execution.Implicits._
object Application extends Controller {
def social = Action {
Async {
val twitterPromise = WS.url("http://search.twitter.com/search.json").withQueryString(("q", "playframework")).get()
val githubPromise = WS.url("https://api.github.com/legacy/repos/search/playframework").get()
for {
twitterResponse <- twitterPromise
githubResponse <- githubPromise
} yield Ok(views.html.social.render((twitterResponse.json \\ "text").map(_.as[String]), (githubResponse.json \\ "name").map(_.as[String])))
}
}
}
</script>
<p>Then display the results with the following render.scala.html page:</p>
<script class="brush:java" type="syntaxhighlighter">
@(tweets: Seq[String], repos: Seq[String])
<h2>Tweets</h2>
<p>
<ul>@for(tweet <- tweets) {
<li>@tweet</li>
}
</ul>
</p>
<h2>Github Repos:</h2>
<p>
<ul>@for(repo <- repos) {
<li>@repo</li>
}
</ul>
</script>
<p>At the beginning of the application, I got the following error:</p>
<script class="brush:java" type="syntaxhighlighter">
cannot find an implicit ExecutionContext
</script>
The solution is to import the following:
<script class="brush:java" type="syntaxhighlighter">
import play.api.libs.concurrent.Execution.Implicits._
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com1tag:blogger.com,1999:blog-9150005423109009329.post-51984698316239632092013-02-20T14:56:00.000-08:002013-02-20T15:16:53.056-08:00Working With Nested Forms with Play 2.1 When creating nested forms and modifying input parameters into a Play 2.1 application seems simple, in many occasions it can be confusing. Here is an application with a model named Hedger which has a object named MarketData. In here I will try to explain the following:
<ul>
<li>How to handle a nested form inside the controller
<li>How to configure the nested form in the view page
<li>Create custom validations with Play 2.1 form
</ul>
<p>
The first thing that we will show is the model objects - Hedger and MarketData
</p>
<script class="brush:java" type="syntaxhighlighter">
case class MarketData(quantity: Double, futurePrice: Double, volatility: Double)
case class Hedger(id: Option[Long], email: String, rate: Double, riskPrice: Double,
correlationPrice: Double, sixMonth: MarketData, year: MarketData, created: Option[Date])
</script>
<p>Now, we will create the form inside the controller</p>
<script class="brush:java" type="syntaxhighlighter">
private val hedgerForm = Form(
mapping(
"id" -> optional(longNumber),
"email" -> email,
"rate" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"riskPrice" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"correlationPrice" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"sixMonth" -> mapping(
"quantity" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"futurePrice" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"volatility" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_))
)
((quantity, futurePrice, volatility) => MarketData(quantity.toDouble, futurePrice.toDouble, volatility.toDouble))
((data: MarketData) => Some(data.quantity.toString, data.futurePrice.toString, data.volatility.toString)),
"year" -> mapping(
"quantity" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"futurePrice" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"volatility" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_))
)((quantity, futurePrice, volatility) => MarketData(quantity.toDouble, futurePrice.toDouble, volatility.toDouble))
((data: MarketData) => Some(data.quantity.toString, data.futurePrice.toString, data.volatility.toString)),
"created" -> optional(date)
) ((id, email, rate, riskPrice, correlationPrice, sixMonth, year, created) =>
Hedger(id, email, rate.toDouble, riskPrice.toDouble, correlationPrice.toDouble, sixMonth, year, created))
((hedger: Hedger) => Some(hedger.id, hedger.email, hedger.rate.toString, hedger.riskPrice.toString, hedger.correlationPrice.toString, hedger.sixMonth, hedger.year, hedger.created))
)
</script>
<p>Noticed that there is a "sixMonth" and a "year" field in the form. These two fields are bind with the MarketData object. Therefore, we need to bind the input parameters into the case class. However, the object accepts Double and the form has "text". To handle this, we used a custom validation by using "text.verifying". The verifying contains two parameters:
<ul>
<li>Error message</li>
<li>Boolean validation that there is an error - isInvalid</li>
</ul>
</p>
<script class="brush:java" type="syntaxhighlighter">
"sixMonth" -> mapping(
"quantity" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"futurePrice" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_)),
"volatility" -> text.verifying("hedger.validation.invalid.number", Hedger.validateDouble(_))
)
((quantity, futurePrice, volatility) => MarketData(quantity.toDouble, futurePrice.toDouble, volatility.toDouble))
((data: MarketData) => Some(data.quantity.toString, data.futurePrice.toString, data.volatility.toString)),
</script>
<p>The Hedger.validateDouble has the following code</p>
<script class="brush:java" type="syntaxhighlighter">
object Hedger {
def validateDouble(value: String): Boolean = {
value.matches("(^\\d+|^\\d+\\.\\d+$)")
}
}
</script>
<p>You can also do the match inside the form, but I did this to make the code cleaner and to show that you can do the checking inside the object</p>
<p>When binding the object, you need to take care of the "apply" and "unapply". In the apply section, you need to show the mapping of the Hedger object, but since this is a nested form, you also need to take care of the MarketData as we saw in the previous code.
</p>
<p>Here is the view page for the application:
</p>
<script class="brush:java" type="syntaxhighlighter">
<div class="contact-info span12">
<h1>@Messages("hedging.form.title") <small>all fields are required</<small></h1>
@helper.form(action = routes.HedgingModels.calculate, 'id -> "validForm") {
<div class="row show-grid">
<fieldset>
<input type="hidden" name="email" value='bwu@@guzman.com' >
<div class="span4">
@helper.inputText(hedgerForm("rate"),
'_label -> Messages("hedging.form.rate.title"),
'_value -> "0.002")
</div>
<div class="span4">
@helper.inputText(hedgerForm("riskPrice"),
'_label -> Messages("hedging.form.riskprice.title"),
'placeholder -> "0.5")
</div>
<div class="span4">
@helper.inputText(hedgerForm("correlationPrice"),
'_label -> Messages("hedging.form.correlation.title"),
'placeholder -> "0.66")
</div>
</fieldset>
</div>
<!-- START SIX AND YEAR QUANTITY PRODUCTED -->
<div class="row show-grid">
<div class="contact-info span6">
<h2>Six Month</h2>
<fieldset>
@inputText(hedgerForm("sixMonth.quantity"),
'_label -> Messages("hedging.form.quantity.title"),
'placeholder -> "100")
@inputText(hedgerForm("sixMonth.futurePrice"),
'_label -> Messages("hedging.form.futureprice.title"),
'placeholder -> "3.43")
@inputText(hedgerForm("sixMonth.volatility"),
'_label -> Messages("hedging.form.volatility.title"),
'placeholder -> "0.31")
</fieldset>
</div>
<div class="contact-info span6">
<h2>One Year</h2>
<fieldset>
@inputText(hedgerForm("year.quantity"),
'_label -> Messages("hedging.form.quantity.title"),
'placeholder -> "100")
@inputText(hedgerForm("year.futurePrice"),
'_label -> Messages("hedging.form.futureprice.title"),
'placeholder -> "3.43")
@inputText(hedgerForm("year.volatility"),
'_label -> Messages("hedging.form.volatility.title"),
'placeholder -> "0.31")
</fieldset>
</div>
</div>
<p><input type="submit" class="btn primary" value='@Messages("hedging.form.submit.button")'></p>
}
</div>
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-46196148657203743142013-02-07T18:21:00.001-08:002013-02-11T18:46:36.075-08:00Scala problems - using List<p>Trying to better my Scala programming by trying some examples/questions of the rich collection API.</p>
<p>How to shuffle a list of strings:</p>
<script class="brush:java" type="syntaxhighlighter">
def math_vocabulary = List("capacity", "gallon", "gram", "length", "liter", "mass", "meter", "ounce", "pound", "ton", "weight", "decimiter", "dekameter", "millileter", "milimeter", "percision")
util.Random.shuffle((0 to math_vocabulary.length -1).toList).map((i:Int) => math_vocabulary(i))
res25: List[java.lang.String] = List(gram, dekameter, length, liter, meter, ounce, millileter, gallon, milimeter, percision, mass, weight, capacity, pound, ton, decimiter)
</script>
<script src='http://syntaxhighlight-blogger-dynamic.googlecode.com/files/syntaxhighlighter-20111212.js' type='text/javascript'></script>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-67866670015477444932013-02-05T18:50:00.002-08:002013-02-11T18:46:16.870-08:00Problems brew install - could not link [app]<p>My favorite command to install application in mac is <a href="http://mxcl.github.com/homebrew/">homebrew</a>. Today I was getting the following error when installing scala:</p>
<script class="brush:java" type="syntaxhighlighter">
$ brew install scala
==> Downloading http://www.scala-lang.org/downloads/distrib/files/scala-2.9.2.tg
Already downloaded: /Library/Caches/Homebrew/scala-2.9.2.tgz
==> Downloading https://raw.github.com/scala/scala-dist/27bc0c25145a83691e3678c7
######################################################################## 100.0%
==> Caveats
Bash completion has been installed to:
/usr/local/etc/bash_completion.d
Warning: Could not link scala. Unlinking...
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
You can try again using `brew link scala'
</script>
<p>The solution is to make the directory as a member of my username</p>
<script class="brush:java" type="syntaxhighlighter">
$sudo chown -R [your-username] /usr/local/
</script>
<script src='http://syntaxhighlight-blogger-dynamic.googlecode.com/files/syntaxhighlighter-20111212.js' type='text/javascript'></script>
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-9327984034501798972013-02-04T14:12:00.000-08:002013-02-05T09:56:09.558-08:00Send e-mail using Play 2, Scala, and Emailer pluggin<p>I needed to send an e-mail when a user fills up a contact form. Below is the how I implemented the application using the <a href="https://github.com/typesafehub/play-plugins/tree/master/mailer">mailer</a> plugin.</p>
<p>First, we add the e-mailer library: projects/Build.scala</p>
<script class="brush:java" type="syntaxhighlighter">
val appDependencies = Seq(
"com.typesafe" %% "play-plugins-mailer" % "2.0.4"
)
</script>
<p>Then, we add the play plugin conf/play.plugins:</p>
<script class="brush:java" type="syntaxhighlighter">
1500:com.typesafe.plugin.CommonsMailerPlugin
</script>
<p>Now, you need to add the smtp host in conf/application.conf:</p>
<script class="brush:java" type="syntaxhighlighter">
smtp.host=enterprisealerts.corp.acme.com
</script>
<p>Now, we add the service:</p>
<script class="brush:java" type="syntaxhighlighter">
package service
import models.Contact
import com.typesafe.plugin._
import play.api.i18n.Messages
import play.api.Play.current
object EmailService {
def sendEmail(contact: Contact) = {
val mail = use[MailerPlugin].email
mail.setSubject(Messages("contact.request.subject", contact.email))
mail.addRecipient(Messages("contact.request.email.recipient"))
mail.addFrom(Messages("contact.request.email.from"))
mail.send(Messages("contact.request.body", contact.email, contact.name, contact.phone, contact.comments))
}
}
</script>
From the controller, then I access this service:
<script class="brush:java" type="syntaxhighlighter">
def create() = Action { implicit request =>
contactForm.bindFromRequest.fold(
hasErrors = { form =>
Redirect(routes.ContactUs.newContact()).flashing(Flash(form.data) +
("error" -> Messages("contact.validation.errors")))
},
success = { newContact =>
Contact.create(newContact)
EmailService.sendEmail(newContact)
val message = Messages("contact.new.success", newContact.email)
Redirect(routes.ContactUs.show(newContact.email)).flashing("success" -> message)
}
)
}
</script>
<p>It is using the Contact model which has the following fields:</p>
<script class="brush:java" type="syntaxhighlighter">
case class Contact(id: Option[Long], email: String, name: String, phone: String, comments: Option[String], signedup: Option[Date])
</script>
<p>The application needs to import the following:</p>
<script class="brush:java" type="syntaxhighlighter">
import play.api.Play.current
import com.typesafe.plugin._
</script>
<p>Then, you can start building you e-mail service by using:</p>
<script class="brush:java" type="syntaxhighlighter">
val mail = use[MailerPlugin].email
mail.setSubject(Messages("contact.request.subject", newContact.email))
mail.addRecipient(Messages("contact.request.email.recipient"))
mail.addFrom(Messages("contact.request.email.from"))
mail.send(Messages("contact.request.body", newContact.email, newContact.name, newContact.phone, newContact.comments))
</script>
Where all the messages are stores in the conf/messages
<script class="brush:java" type="syntaxhighlighter">
#Contact Us e-mail request/confirmation
contact.request.email.recipient=test@acme.com
contact.request.email.from=noreply@acme.com
contact.request.subject=Contact request from {0}
contact.request.body=Contact request from: e-mail: {0}, name: {1}, phone: {2}, comment: {3}
</script>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com1tag:blogger.com,1999:blog-9150005423109009329.post-31259451905898237822013-02-01T19:19:00.004-08:002013-02-04T07:48:46.901-08:00Using forms with Play framework 2 and Twitter's Bootstrap<P>I've been trying to learn <a href="http://twitter.github.com/bootstrap/">Twitter's Bootstrap</a>, <a href="http://www.playframework.org/">Play2</a>, and <a href="http://www.scala-lang.org/">Scala</a> for the past month. The past two days, I've been trying to create a basic HTML application along with a controller, model, persistant layer, and HTML form. It took me a bit of time to really understand what was going on, but I think I finally got it. The form is a basic "Contact Us" and here are the requirements:</p>
<ul>
<li>User should prompt to enter a contact form: name, e-mail, phone, and comment</li>
<li>Form should validated any errors and report a friendly message in case of such error</li>
<li>Display a message back to the user that the information was indeed saved</li>
</ul>
<p>Here is what my controllers will have:</p>
<ul>
<li>A form with all the validation</li>
<li>A controller that will take the user to the form page</li>
<li>A controller that will validate the form, and either send it back to the form's page in case of an error, or display a successful messages in the details page</li>
<li>A controller that will act as an "edit" page, which displays the form information successfully saved</li>
</ul>
<p>First, lets start with the form. We have a "contactForm" which only needs to have: </p>
<ul>
<li>Client's name - required field</li>
<li>Client's e-mail - required field and validates if it's already in our contact list</li>
<li>Client's phone number - required field</li>
<li>Comments - anything that the client can tell us about his/her firm and reason of reaching us</li>
</ul>
<p>However, the model that we have, needs to have more than these fields. For example, we need to have an ID, and also a signed-up date that tell us when the customer registered with us.</p>
<p>The model application will look like this:</p>
<script class="brush:java" type="syntaxhighlighter">
case class Contact(id: Long, email: String, name: String, phone: String, comments: String, signedup: Date)
</script>
<p>However, because the id, comments, and signedup fields are not required, we need to add Scala's "Option" field. The reason of this wrapper is to avoid "nulls". This tells the application that the field is not required and that it might be null. This will also come in hand with our SQL and with the form fields in the controller. Here is how the model will look like:</p>
<script class="brush:java" type="syntaxhighlighter">
case class Contact(id: Option[Long], email: String, name: String, phone: String, comments: Option[String], signedup: Option[Date])
</script>
<p>Now, we need to take care of the persistent layer (DAO/Repos). This layer will take care of displaying all the contacts, retrieve contact by e-mail, and insert a contact into our database.</p>
<p>For this example, I will be using the h2 database provided by the Play framework. You need to comment the following lines in the conf/application.conf</p>
<script class="brush:java" type="syntaxhighlighter">
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:mem:play"
</script>
<p>Since, I will be using <a href="https://github.com/playframework/Play20/wiki/ScalaAnorm">Anorm</a>, we need to create the SQL script for your SQL <a href="https://github.com/playframework/Play20/wiki/Evolutions">evolutions</a>. You need to create a folder "conf/evolutions/default" and add a file 1.SQL with the following code:</p>
<script class="brush:java" type="syntaxhighlighter">
# --- First database schema
# --- !Ups
CREATE TABLE contacts (
id SERIAL PRIMARY KEY,
name VARCHAR(100) NOT NULL,
email VARCHAR(50) NOT NULL,
phone VARCHAR(25) NOT NULL,
comments VARCHAR(250),
signedup TIMESTAMP
);
ALTER TABLE contacts ADD CONSTRAINT EMAIL_UNIQUE UNIQUE(email)
# --- !Downs
DROP TABLE IF EXISTS contacts;
</script>
Now, we can create the rest of the model:
<script class="brush:java" type="syntaxhighlighter">
package models;
import play.api.db._
import play.api.Play.current
import anorm._
import anorm.SqlParser._
import java.util.Date
case class Contact(id: Option[Long], email: String, name: String, phone: String, comments: Option[String], signedup: Option[Date])
object Contact {
val simple = {
get[Option[Long]]("id")~
get[String]("email")~
get[String]("name")~
get[String]("phone")~
get[Option[String]]("comments")~
get[Option[Date]]("signedup") map {
case id~email~name~phone~comments~signedup => Contact(id,email,name,phone,comments,signedup)
}
}
def findAll(): Seq[Contact] = {
DB.withConnection { implicit connection =>
SQL("select * from contacts order by signedup").as(Contact.simple *)
}
}
def findByEmail(email: String): Option[Contact] = {
DB.withConnection { implicit connection =>
SQL("select * from contacts where email = {email}")
.on('email -> email)
.as(Contact.simple *)
.headOption
}
}
def create(contact: Contact): Unit = {
DB.withConnection { implicit connection =>
SQL("insert into contacts(name, email, phone, comments, signedup) values({name},{email},{phone},{comments},{signedup})").on(
'name -> contact.name,
'email -> contact.email,
'phone -> contact.phone,
'comments -> contact.comments,
'signedup -> new Date()
).executeUpdate()
}
}
}
</script>
<p>Inside the object method, you will find the simple mapper. This is simply mapping each of the SQL record. It is <strong>important to be consistent with the case class</strong>. The method's purpose can be seen in the findAll implementation. The application will respond with a sequence of contacts, by executing an SQL statement, and calling the simple to all records (hence the "*") to be map using the "simple" mapping.</p>
<p>The findByMail is a bit different. The main purpose of the method is to find out if there are no records for this e-mail. To avoid null or empty validations, it's recommended to use Scala's "Option" wrapper. This is why we will add the Option[Contact] as our return statement, and the "headOption" when returning the response. This will be very helpful for our form as we will see next. Otherwise, you will get errors similar to this one:</p>
<script class="brush:java" type="syntaxhighlighter">
type mismatch; found : anorm.ResultSetParser[List[models.Contact]] required: anorm.ResultSetParser[Option[models.Contact]]
</script>
<p>Now that we have our model and persistent layer, here is the controller code:</p>
<script class="brush:java" type="syntaxhighlighter">
package controllers
import play.api._
import play.api.mvc._
import play.api.data.Form
import play.api.data.Forms.{mapping, nonEmptyText, email, optional, text, date, longNumber}
import play.api.i18n.Messages
import models.Contact
object ContactUs extends Controller {
private val contactForm: Form[Contact] = Form(
mapping(
"inputId" -> optional(longNumber),
"email" -> email.verifying(
"contact.validation.email.duplicate", Contact.findByEmail(_).isEmpty),
"name" -> nonEmptyText,
"phone" -> nonEmptyText,
"comment" -> optional(text),
"signedup" -> optional(date)
)(Contact.apply)(Contact.unapply))
//Create a new contact
def newContact = Action { implicit request =>
val form = if (flash.get("error").isDefined)
this.contactForm.bind(flash.data)
else
this.contactForm
Ok(views.html.contactus.contact(form))
}
//Display the contact by the e-mail
def show(email: String) = Action { implicit request =>
Contact.findByEmail(email).map { contact =>
Ok(views.html.contactus.details(contact))
}.getOrElse(NotFound)
}
//Create the form or send out error to user depending on the form
def create() = Action { implicit request =>
contactForm.bindFromRequest.fold(
hasErrors = { form =>
Redirect(routes.ContactUs.newContact()).flashing(Flash(form.data) +
("error" -> Messages("contact.validation.errors")))
},
success = { newContact =>
Contact.create(newContact)
val message = Messages("contact.new.success", newContact.email)
Redirect(routes.ContactUs.show(newContact.email)).flashing("success" -> message)
}
)
}
}
</script>
<p>As you can see, the form will be doing the validation for each field. Here's is where I failed: I was under the impression that I should only have the fields that were in my form. It so happens, that the Contact.apply and Contact.unapply is mapped to the case class; therefore, you need to specify all the fields. This type of error caused the following message:</p>
<script class="brush:java" type="syntaxhighlighter">
type mismatch; found : (anorm.Pk[Long], String, String, String, Option[String], Option[java.util.Date]) => models.Contact required: (String, String, String, Option[String]) => ?
</script>
<p>Now, for the validation, we need to add is the id as optional since this will be given by the DB when creating the record. Next is the e-mail validation. As requested, there needs to be a validation in case the e-mail has been recorded previously, and return a friendly error message. The email calls a "verifying" method that takes two parameters. One is the error message in the case that the field is not valid. The second parameter is a boolean that checks the validity of the value. In the case of the e-mail, we check if it's in the database - "Contact.findByEmail(_).isEmpty". The "isEmpty" method is provided by "Option" wrapper - in case if it's null or empty, it will be true.</p>
<p>The "newContact" controller will take care of displaying the form. It also sends the form back in case there are any errors. This is why we add the "implicit request". We will be using the flash for a temporary storage of the error.</p>
<p>Most modern web-frameworks have a flash-scope. Like the session-scope it is meant to keep data, related to the client, outside of the context of a single request. The difference is that the flash-scope is kept for the next request only, after which it's removed. This takes some effort away from you, as the developer, because you don't have to write code that clears things like one-time message from the session.</p>
<p>Play implements this in the form of a cookie that's cleared on every response, except for the response that sets it. The reason for using a cookie is scalability. If the flash is not stored in the server, each of one of a client's requests can be handled by a different server, without having synchronize between servers.The session is kept in a cookie for exactly the same reason. This makes setting a cluster a lot simpler. You don't need to send a particular client's request to the same server, you can simply hand out requests to servers on a round-robin basis.</p>
<p>Now, lets look at the "create" controller. This will be the action called by the form. The application will get the HTTP request, and bind the request. Then, it will call the fold command to see if the form values have errors or if it's successful. In Scala "fold" is often used as the name of a method that collapses (or folds) multiple possible values into a single value. The fold method has two parameters, both of which are functions. So, "hasErrors" is called if validation failed, and "success" if it validates without errors</p>
<p>Below is the contact.scala.html code:</p>
<script class="brush:java" type="syntaxhighlighter">
@(contactForm: Form[Contact])(implicit flash: Flash)
@import helper.twitterBootstrap._
@import helper._
@implicitFieldConstructor = @{ FieldConstructor(contacterror.render) }
@main(Messages("company.full.title"), Messages("contactus.title")) {
<!-- MAIN CONTENT AREA -->
<div class="main-content">
<div class="container">
<div class="row show-grid">
<div class="span12" data-original-title="">
<!-- BREADCRUMBS -->
<div id="breadcrumb">
<ul>
<li class="home"><a href="#">Home</a></li>
<li>@Messages("contactus.title")</li>
</ul>
</div>
<div class="clear_both"></div>
<div class="col-wrapper">
<div class="row show-grid">
<!-- START LEFT-SIDE CONTACT FORM AREA -->
<div class="contact-info span8 ">
@helper.form(action = routes.ContactUs.create, 'id -> "validForm") {
<fieldset>
<legend>Contact Information</legend>
@helper.inputText(contactForm("name"),
'_label -> Messages("contactus.form.name"),
'_help -> "Please enter your name.",
'_class -> "span4",
'placeholder -> "Name")
@helper.inputText(contactForm("email"),
'_label -> Messages("contactus.form.email"),
'_help -> "Please enter a valid e-mail address.",
'placeholder -> "E-mail")
@helper.inputText(contactForm("phone"),
'_label -> Messages("contactus.form.phone"),
'_help -> "Please enter a phone where we can reach you.",
'placeholder -> "Phone")
@helper.textarea(contactForm("comment"),
'_label -> Messages("contactus.form.comment"),
'class -> "span8",
'placeholder -> "Comment")
</fieldset>
<p><input type="submit" class="btn primary" value='@Messages("contactus.form.submit.button")'></p>
}
</div>
<!-- END LEFT-SIDE CONTACT FORM AREA -->
<!-- BEGIN RIGHT-SIDE CONTACT FORM AREA -->
<div class="contact-info span4">
<h2>@Messages("company.full.title")</h2>
<!-- COMPANY ADDRESS -->
@views.html.contactus.address()
</div>
<!-- END RIGHT-SIDE CONTACT FORM AREA -->
</div>
</div>
</div>
</div>
<hr>
</div>
</div>
}
</script>
<p>As I mentioned at the beginning, I'm using Twitter's Bootstrap to handle all the CSS and HTML5 goodness. Therefore, I want to leverage as much as possible all of that. To activate the CSS handling and using the helper functions for Play you need to import a few fields. Also, you need to add the form and flash scope parameter to the HTML. I also used the implicitFieldConstructor to show where the errors happened. We will show that later. First lets emphasize on the form. The actions for the form will be handle by "@helper.form(action = routes.ContactUs.create, 'id -> "validForm")". Then, we will add the fields by using the "fieldset". I wanted to leverage the uses of place holder, labels, and classes on all the input fields. However, the most important part is to know that @helper.inputText(contactForm("name")) consists on the name of the input. The id goes inside the "contactForm".</p>
<p>I also provided some type of error friendly validation in case the user has some erroneous fields. I created a "contacterror.scala.html" with the following code:</p>
<script class="brush:java" type="syntaxhighlighter">
@(elements: helper.FieldElements)
<div class="@if(elements.hasErrors) {alert alert-error}">
<label for="@elements.id">@elements.label</label>
<div class="input">
@elements.input
<span class="errors">@elements.errors.mkString(", ")</span>
<span class="help">@elements.infos.mkString(", ")</span>
</div>
</div>
</script>
<p>Then, I added the @implicitFieldConstructor = @{ FieldConstructor(contacterror.render) } in the form (contact.scala.html). Again, this way, we will see the highlights and the friendly errors on the fields.
</p>Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com3tag:blogger.com,1999:blog-9150005423109009329.post-21016804400935034062013-01-29T14:11:00.002-08:002013-01-30T10:08:13.202-08:00Using templates with Play 2 and Scala<p>I admit, it has been a while since I have done web development. To be more precise, the last framework I used was Struts 1.x (back in early 2000's). It was the de facto MVC framework of its time, but I end up loving it when I found <a href="http://struts.apache.org/1.x/struts-tiles/index.html">tiles</a>. Tiles introduced the template format, and is one reason why I end up using <a html="http://www.playframework.org/">Play 2</a> along with Scala in my new web applications. The use of templates is something that I really enjoy - makes my job a lot easier and more productive.</p>
<p>Lets say that you have a series of products to display. Your Play controller will look at something like this:</p>
<script class="brush:java" type="syntaxhighlighter">
def catalog() = Action {
val products = ProductDAO.list
Ok(views.html.shop.catalog(products))
}
</script>
Now, lets assume that the app/views/catalog.scala.html contains the following:
<script class="brush:java" type="syntaxhighlighter">
@(products: Seq[Product])
<!DOCTYPE html>
<html>
<head>
<title>paperclips.example.com</title>
<link href="@routes.Assets.at("stylesheets/main.css")"
rel="stylesheet">
</head>
<body>
<div id="header">
<h1>Product catalog</h1>
</div>
<div id="navigation">
<ul>
<li><a href="@routes.Application.home">Home</a></li>
<li><a href="@routes.Shop.catalog">Products</a></li>
<li><a href="@routes.Application.contact">Contact</a></li>
</ul>
</div>
<div id="content">
<h2>Products</h2>
<ul class="products">
@for(product <- products) {
<li>
<h3>@product.name</h3>
<p class="description">@product.description</p>
</li>
}
</ul> </div>
<footer>
<p>Copyright 2013 play.example.com</p>
</footer>
</body>
</html>
</script>
The code will show all the products, but also it shows the navigation, and footer. Clearly, we want to separate these contents so we can reuse them in another application. Furtheremore, in case that the footer has all the javascripts and Google analytics, we need to make sure that indeed <i>all</i> pages get tracked.
<p>Using Play 2, you can simply extract all your navigation to a file name app/views/navigation.scala.html that will contain the navigation code:</p>
<script class="brush:java" type="syntaxhighlighter">
@()
<div id="navigation">
<ul>
<li><a href="@routes.Application.home">Home</a></li>
<li><a href="@routes.Shop.catalog">Catalog</a></li>
<li><a href="@routes.Application.contact">Contact</a></li>
</ul> </div>
</script>
Then, do the same thing with the fotter app/views/footer.scala.html:
<script class="brush:java" type="syntaxhighlighter">
@()
<footer>
<p>Copyright 2013 play.example.com</p>
</footer>
</script>
Now, to show the contents of the navigation and the footer, simply use the code @navigation() and @footer() to show the contents. The catalog.scala.html will now be like this:
<script class="brush:java" type="syntaxhighlighter">
<!DOCTYPE html>
<html>
<head>
<title>paperclips.example.com</title>
<link href="@routes.Assets.at("stylesheets/main.css")"
rel="stylesheet">
</head>
<body>
<div id="header">
<h1>Products</h1>
</div>
@navigation()
<div id="content">
// Content here
</div>
@footer()
</body>
</html>
</script>
<p>The other reason that I really enjoy Play 2 with Scala is the reverse routing. Reverse routing is a way to programmatically access the routes configuration, to generate a URL for a given action method invocation. In other words, you can do reverse routing by writing Scala code!</p>
<p>As you can see in the navigation, I don't have any hard coded routes. For example, home has the link: "@routes.Application.home". This is perfect in case of refactoring! My routes for home and catalog will depend on the configuration of my routes (contained in the conf directory):</p>
<script class="brush:java" type="syntaxhighlighter">
GET / controllers.Application.home
GET /shop controllers.Shop.catalog
</script>
<p>If tomorrow I want to change the path, that will not affect my code at all, just the route file.</p>
<p>Again, this is an efficient way of programming because you can leverage the templates to build a user-interface view and you can use user-friendly URL with the help of your routes.Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com0tag:blogger.com,1999:blog-9150005423109009329.post-15765767439982939362013-01-28T09:04:00.000-08:002013-01-28T09:07:21.316-08:00Understanding the implicit on classesWhen I was learning Scala I stumbled into the implicit keyword. It took me a bit to understand it, but I really got into it once I started using Play. The best way that I can explain it is by thinking about "extending" the class without actually changing the code. Here are some examples:
<script class="brush:java" type="syntaxhighlighter">
scala> case class Person(firstName: String, lastName: String, age: Int)
defined class Person
scala> class PersonWrapper(p: Person) {
| def minor = p.age < 18
| def adult = !minor
| }
defined class PersonWrapper
scala> implicit def personWrapper(p: Person) = new PersonWrapper(p)
personWrapper: (p: Person)PersonWrapper
scala> Person("Marcelo", "Olivas", 36).adult == true
res0: Boolean = true
scala> Person("Andres", "Olivas",12).minor == true
res1: Boolean = true
</script>
This type of code is very useful. If you are using the Play framework, then you probably seem this type of code:
<script class="brush:java" type="syntaxhighlighter">
object Products extends Controller {
def list = Action { implicit request =>
val products = Product.findAll
Ok(views.html.products.list(products))
}
}
</script>
In here, we are implicitly extending the HTTP request so we can inject a product list.
This type of code is very useful because you can still have the Person class without been compromised (immutable/intact).
The same goes for the example below:
<script class="brush:java" type="syntaxhighlighter">
object Product {
import anorm.SQL
import anorm.SqlQuery
val sql: SqlQuery = SQL("select * from products order by name asc")
import play.api.Play.current
import play.api.db.DB
def findAll: List[Product] = DB.withConnection { implicit connection =>
sql().map ( row =>
Product(row[Long]("id"), row[Long]("ean"),
row[String]("name"), row[String]("description"))
).toList
}
</script>
Using Anorm, we use implicit to use the SQL connection.
However, as you can imagine, this can also have some problems. You can read more about it <a href="http://twitter.github.com/effectivescala/#Types%20and%20Generics-Implicits">here</a>.
Marcelo Olivashttp://www.blogger.com/profile/10776704558286204808noreply@blogger.com3