Drought Hack: Washing Machine Drain to Garden Hose

855c1ef720551b71a4d89cbb1a401374a2c9b141

Sun 12 Apr 2015 13:08:03 -0800

With summer looming on the horizon and the California drought becoming an ever increasing problem, water conservation is undoubtedly a concern if not a necessity. Faced with a mandatory 25% water consumption reduction imposed by the State of California, my wife and I decided to seriously look at how we could further reduce our water usage — and for the water we must use, how we could reuse as much as possible where it made sense. Unfortunately, as a couple who already self-imposed deep water restrictions in mid-2014, the State’s latest mandatory 25% reduction on top of what we were already doing to conserve made the government’s ask feel all the more egregious.

And so, here we are.

Looking around the house for ways to even further conserve and reuse water, we stumbled across the ole’ washing machine. A classic Kenmore 500, this bad boy is clearly a big consumer of household water — fill up, wash, rinse, repeat. Studying the machine and its operation, my wife and I noticed just how much relatively clean greywater we were pumping down the drain during each wash cycle.

The light bulbs above both of our heads went on almost in unison — lets capture that water for the yard!

A quick trip to the hardware store, and bingo: ten minutes and $11 later we had the parts needed to send our washing machine greywater water into a garden hose.

Parts

Your washing machine, and garden hose, will almost certainly be different so you should get a tape measure and identify the right drain hose dimensions for your washing machine.

Here’s what we used:

  1. Flexible Rubber Pipe Coupler with Clamps


  2. PVC Schedule 40, Slip x NPT Female Adapter


  3. PVC Schedule 40, NPT Male x MHT Male Adapter



  4. Garden Hose


Assembly

With two hands, a wrench, and a gentle but firm touch you should be able to assemble these parts like so:



Some PVC glue for the slip to NPT adapter, and vinyl plumbers tape for the threads is good too, if you’re into that sort of thing.

Winning

And there you have it: a relatively cheap and clever solution to capture greywater water from your washing machine to be used in/around your yard.

Detergent

We did some digging and from what we can tell, mild detergents are not a problem for most ornamental plants and trees — of course, environmentally friendly detergents that degrade easily are best. And, it should go without saying, but chlorine bleach/borax is harmful for any living thing so you should avoid draining the water from bleached loads onto your yard.

Our plants love it — my wife and I have been watering our yard using this method for several weeks now, and thus far, everything is healthy. Despite our fears, not a single plant has keeled over and died (yet) being watered with washing machine greywater.

Caveats

  1. I take no responsibility and hold no liability if, for whatever reason, you break your washing machine or kill the plants in your yard using greywater from your washing machine. Do your own homework, any mistakes are on you.
  2. Some cities/municipalities have laws regarding greywater usage in/around homes. Before you try any of this, you should consult with your city officials to make sure it’s OK to water your yard with greywater.
  3. It should go without saying, but the water that comes out of your washing machine is non-potable. Meaning you cannot, and should not, drink it. Keep thirsty pets and animals away too.
  4. Don’t try to store greywater — in a 50-gallon drum, garbage can, etc. It’s gross. Apparently if you want to store greywater, there are advanced systems you can buy that will let you do exactly that. Storing untreated greywater from a washing machine in a 50-gallon drum is just asking for trouble.

Happy watering!

A Quick Bash Script to Backup Files on AWS S3

70464114a3dbcc5d85eb3fe0ba1d1f12bd24c97c

Sun 11 Jan 2015 13:08:26 -0800

Last week, I lost a disk in a 4TB software RAID5 array mounted in my home Linux server. This host has been online for almost 4-years without any major interruptions so the clock was ticking — it was really only a matter of time until I would be forced to replace a disk. Fortunately, replacing the disk in the array was a complete breeze. No data was lost, and rebuilding the array with a new disk only took a short 160-minutes while the host remained online. Hats off to the folks maintaining Linux RAID — the entire disk replacement process end-to-end, was flawless.

Before I could replace the failing disk, the array was limping along in a degraded state. This got me thinking: I already regularly backup the data I cannot live without to an external USB pocket drive and store it “offsite” — what if I could sync the most important stuff to the “cloud” too?

So, I sat down and wrote a quick bash script that recursively crawls a root directory of my choosing and uses curl to upload each discovered file to AWS S3. Note that the structure of the backup on S3 will exactly match the file/directory structure on disk:

#!/bin/bash

S3_KEY="[YOUR AWS KEY HERE]"
S3_SECRET="[YOUR AWS SECRET HERE]"

BUCKET="[YOUR BUCKET NAME HERE]"

CONTENT_TYPE="application/octet-stream"

find $@ -type f -print0 | while IFS= read -r -d '' i; do
  FILE="$(perl -MURI::Escape -e 'print uri_escape($ARGV[0],"^A-Za-z0-9\-\._~\/");' "$i")"
  RESOURCE="/${BUCKET}/${FILE}"
  DATE_VALUE=`date -R`
  STRING_TO_SIGN="HEAD\n\n\n${DATE_VALUE}\n${RESOURCE}"
  SIGNATURE=`echo -en ${STRING_TO_SIGN} | openssl sha1 -hmac ${S3_SECRET} -binary | base64`
  EXISTS=`curl -s -I -w "%{http_code}" \
    -o /dev/null \
    -H "Host: ${BUCKET}.s3.amazonaws.com" \
    -H "Date: ${DATE_VALUE}" \
    -H "Authorization: AWS ${S3_KEY}:${SIGNATURE}" \
    https://${BUCKET}.s3.amazonaws.com/${FILE}`
  if [ $EXISTS -eq "200" ];
  then
    echo "File \"$i\" exists."
  else
    echo $i
    MD5=`openssl dgst -md5 -binary "$i" | base64`
    STRING_TO_SIGN="PUT\n${MD5}\n${CONTENT_TYPE}\n${DATE_VALUE}\n${RESOURCE}"
    SIGNATURE=`echo -en ${STRING_TO_SIGN} | openssl sha1 -hmac ${S3_SECRET} -binary | base64`
    curl -# -X PUT -T "${i}" \
      --limit-rate 300k \
      --connect-timeout 120 \
      -H "Host: ${BUCKET}.s3.amazonaws.com" \
      -H "Date: ${DATE_VALUE}" \
      -H "Content-Type: ${CONTENT_TYPE}" \
      -H "Content-MD5: ${MD5}" \
      -H "Authorization: AWS ${S3_KEY}:${SIGNATURE}" \
      https://${BUCKET}.s3.amazonaws.com/${FILE} > /dev/null
  fi
done

If you’d rather not copy+paste, download the script here.

A few notes:

  • You should replace S3_KEY, S3_SECRET, and BUCKET in the script with your AWS key, AWS secret, and backup bucket name respectively.
  • I’m using the --limit-rate 300k argument to limit the upload speed to 300 KB/sec. Otherwise, I’d completely saturate my upload bandwidth at home. You should, of course, adjust this limit to suit your needs depending on where you’re uploading from.
  • I’m using the --connect-timeout 120 argument to work around spurious connection failures that might occur during a handshake with S3 while starting an upload.
  • Documentation on the request signing mechanism used in the script can be found at http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html.

Usage

Assuming you have a directory named foobar which contains a nested structure of the content you want to upload:

chmod +x s3-backup.sh

./s3-backup.sh foobar

Or maybe you only want to upload foobar/baz/*:

./s3-backup.sh foobar/baz

Happy uploading.

Writing Versioned Service APIs With Curacao: Part 2

f577d1b06d808c6f8ec85ba466eae55eab7f4a72

Sun 26 Oct 2014 11:31:30 -0800

In Part 1 of this series, I covered how to use Curacao to handle versioned resource requests. That is, “how do clients specify the version of the resource they’re asking for” given a number of implementation possibilities. In this Part 2, let’s talk routing and a few respective implementation strategies using Curacao.

Routing

How does the API route version specific requests?

Path

Without question, the most common mechanism used to “route” requests to the right controller is through the URI path.

By default, Curacao accomplishes this using regular expressions in conjunction with Java’s named capture groups (in Java 7+) to pull values out of the path as needed. For instance, consider the following RESTful like API requests that manipulate users in a data store.

GET:/users.json?lastName=jones
GET:/user/76234849.json
POST:/user
DELETE:/user/76219057

With a relatively simple set of regular expressions can write a controller that support each of these requests.

import static com.kolich.curacao.annotations.methods.RequestMapping.RequestMethod.*;

@Controller
public final class SampleController {

  private final DataSource ds_;

  @Injectable
  public SampleController(final DataSource ds) {
    ds_ = ds;
  }

  // GET:/users.json?lastName=jones
  // Note the @Query annotation on the 'lastName' argument
  @RequestMapping(value="^\\/users\\.json$", methods={GET})
  public final List<User> getUsers(@Query("lastName") final String lastName) {
    return ds_.getUsersWithLastName(lastName);
  }
  
  // GET:/user/76234849.json
  // Note the @Path annotation on the 'userId' argument
  @RequestMapping(value="^\\/user\\/(?<userId>\\d+)\\.json$", methods={GET})
  public final User getUser(@Path("userId") final String userId) {
    return ds_.getUserById(userId);
  }
  
  // POST:/user
  @RequestMapping(value="^\\/user$", methods={POST})
  public final User createUser(@RequestBody final String createUserRequest) {
    return ds_.createUser(createUserRequest);
  }

  // DELETE:/user/76219057
  @RequestMapping(value="^\\/user\\/(?<userId>\\d+)$", methods={DELETE})
  public final User deleteUser(@Path("userId") final String userId) {
    return ds_.getAndDeleteUser(userId);
  }

}

Nothing too surprising here, but lets walk through it anyways.

This @Controller declares a dependency on DataSource through its constructor — the immutable singleton DataSource will be injected automatically into the constructor when Curacao instantiates an instance of this controller on application startup.

Subsequent methods like getUsers and getUser are only invoked on incoming GET requests whose path matches the regular expression provided in the value attribute of the @RequestMapping annotation.

The @Query controller argument annotation is used to extract the values of query parameters, if any. If @Query references a query parameter that is not present on the request, the argument value will be null. Likewise, the @Path controller argument annotation is used to extract values from the path, if any. If @Path references a named capture group that is not present in the path, or the provided regular expression was unable to extract a value for the given capture group, the argument value will be null.

Beautifully simple — no awful DSL’s to learn, and completely interoperable with other languages for the JVM like Scala and Clojure.

Custom Header

In the odd event that you’d like to route requests based on something other than the path, Curacao supports the implementation of a custom CuracaoPathMatcher to be used within your @RequestMapping annotations. For instance, consider a controller that routes requests based on a custom value within an HTTP request header — it’s easy to implement a custom CuracaoPathMatcher to achieve this behavior.

import com.google.common.collect.ImmutableMap;

public final class MyCustomHeaderMatcher implements CuracaoPathMatcher {

  private static final String MY_CUSTOM_HEADER = "X-Custom-Header";

  @Override @Nullable
  public Map<String,String> match(final HttpServletRequest request,
                                  final String value, // From your @RequestMapping
                                  final String path) throws Exception {
    final String header = request.getHeader(MY_CUSTOM_HEADER);
    if (header != null && header.contains(value)) {
      // If the custom contains the provided value from the annotation,
      // then we have a match!  Note the value argument here is the
      // "value" from the controller method @RequestMapping annotation.
      // For example:
      // @RequestMapping("foo") the value is "foo"
      // @RequestMapping(value="bar", methods=POST) the value is "bar"
      return ImmutableMap.of(MY_CUSTOM_HEADER, value);
    } else {
      return null; // No match!
    }
  }

}

And simply reference your custom CuracaoPathMatcher in your controllers using the matcher attribute on Curacao’s @RequestMapping annoation.

@Controller
public final class SampleController {

  @RequestMapping(value="foo", matcher=MyCustomHeaderMatcher.class)
  public final String foo() {
    // Will only be invoked when an 'X-Custom-Header' request header is
    // present that contains "foo".
    return "foo";
  }
  
  @RequestMapping(value="bar", matcher=MyCustomHeaderMatcher.class)
  public final String bar() {
    // Will only be invoked when an 'X-Custom-Header' request header is
    // present that contains "bar".
    return "bar";
  }

}

In the example above, the foo method will only be invoked when the X-Custom-Header HTTP request header contains the string “foo”. Likewise, the bar method will only be invoked when the X-Custom-Header contains the string “bar”.

You can, of course, implement your own logic to “pull apart” a custom header value and route requests as desired using any custom CuracaoPathMatcher implementation. But, always remember that the first “matcher” to return a non-null map indicating a match, wins. In other words, if you have two custom CuracaoPathMatcher implementations that could potentially match the same “value”, the first matcher that matches will win — the ordering in which matchers are interrogated to find a controller method to invoke is nondeterministic. This is by design.

Part 3

In the upcoming Part 3 of this series, I’ll cover the creation and serving of versioned response objects using Curacao.

Stay thirsty, my friends.

Writing Versioned Service APIs With Curacao: Part 1

00a82f8bb7816500dc79c304a5fb9e58f37116a3

Wed 22 Oct 2014 18:11:19 -0800

Curacao is a beautifully simple toolkit for the JVM that lets you write highly concurrent services on the common and trusted J2EE stack. While you can use it to build a full web-application, Curacao is fundamentally designed to support highly asynchronous REST/HTTP-based integration layers on top of asynchronous Servlets that are easy to understand, maintain, and debug. At its core, Curacao completely avoids the mental overhead of passing of messages between actors or within awful event loops — yet, given its simplicity, performs very well in even the most demanding applications.

Quite often, one of the most difficult problems to solve when designing an API is resource versioning.

As I see it, there’s several aspects to the API versioning problem:

  1. how do clients specify the version of the resource they’re asking for?
  2. how does the API route version specific requests?
  3. how does the API manage and respond with versioned responses?
  4. how does the API gracefully sunset deprecated resource versions?

Solutions to these questions have been debated, ad nauseam, into infinity to which everyone has a conflicting opinion.

Opinions aside, Part 1 of this series highlights a few examples that illustrate how you might implement solutions to these problems with Curacao.

Part 1: Versioned Requests

How do clients specify the version of the resource they’re asking for?

Query Parameter

One approach, is specifying the desired version of a resource using an optional query parameter. For example, consider the following requests:

GET:/user/89171245.json?version=1
GET:/user/89171245.json?version=2
GET:/user/89171245.json

While technically asking for the same resource /user/89171245.json, the client is using the version query parameter to specify the version of the API it intends to use. The server side can interpret the value of the version query parameter, and respond with an entirely unique response object depending on the requested version. In this case, version=1 may result in an entirely different response JSON object compared to that of version=2. In the event that the version query parameter is omitted, the API will default to the most recent version.

Implementing this versioning mechanism with Curacao is trivial.

The trick is to implement a custom ControllerMethodArgumentMapper that looks for the version query parameter, sanitizes it, and passes the requested version to your controller methods as a typed argument.

First, lets define an enumeration that cleanly represents all possible supported API versions.

public static enum MyApiVersion {

  /* API version 1 */
  VERSION_1("1"),
  
  /* API version 2 */
  VERSION_2("2");
  
  private String version_;
  private MyApiVersion(final String version) {
    version_ = version;
  }
  
  /**
   * Given a string, from a query parameter, convert it into one of the
   * supported API versions.  If the param is null, or doesn't match any
   * known version, this method returns the latest version.
   */
  public static final MyApiVersion versionFromParam(final String param) {
    MyApiVersion result = MyApiVersion.VERSION_2; // Default
    if (param != null) {
      // Iterate over each possible version in the enumeration,
      // looking for a match.
      for (final MyApiVersion version : MyApiVersion.values()) {
        if (version.version_.equals(param)) {
          result = version;
          break;
        }
      }
    }
    return result;
  }
  
}

Now, let’s implement a custom ControllerMethodArgumentMapper that converts the version query parameter on the request, if any, into a MyApiVersion.

import com.kolich.curacao.handlers.requests.mappers.ControllerMethodArgumentMapper;

@ControllerArgumentTypeMapper(MyApiVersion.class)
public final class MyApiVersionArgumentMapper extends ControllerMethodArgumentMapper<MyApiVersion> {

  private static final String VERSION_QUERY_PARAM = "version";

  @Nullable @Override
  public final MyApiVersion resolve(@Nullable final Annotation annotation,
                                    final CuracaoRequestContext context) throws Exception {
    final HttpServletRequest request = context.request_;
    final String versionParam = request.getParameter(VERSION_QUERY_PARAM);
    return MyApiVersion.versionFromParam(versionParam);
  }

}

And finally, we can now write our controller methods to take an argument of type MyApiVersion. At runtime, Curacao will see this MyApiVersion argument on your controller methods, and invoke our custom MyApiVersionArgumentMapper to extract the desired version from the request.

import static com.kolich.curacao.annotations.methods.RequestMapping.RequestMethod.*;

@Controller
public final class VersionedController {

  private final DataSource ds_;

  @Injectable
  public VersionedController(final DataSource ds) {
    ds_ = ds;
  }

  @RequestMapping(value="^\\/user\\/(?<userId>\\d+)\\.json$", methods={GET})
  public final String getUser(@Path("userId") final String userId,
                              final MyApiVersion version) {
    final User user = ds_.getUserById(userId);
    if (MyApiVersion.VERSION_1.equals(version)) {
      // Construct and return a "version 1" User object.
    } else {
      // Construct and return a "version 2" User object.
    }
  }
  
}

Note that the MyApiVersion argument in the controller method above is automatically discovered and injected when invoked by Curacao.

Path

A more common approach to request versioning is through the usage of a version identifier in the path itself. For example, consider the following requests:

GET:/v1/user/98143016.json
GET:/v2/user/98143016.json

Note the v1 and v2 version identifier in the path.

Again, unsurprisingly, implementing this versioning mechanism with Curacao is trivial. Current best practices dictate the usage of multiple controllers — one that handles v1 requests and another that handles v2.

And so, one controller for v1:

package com.foo.api.controllers.v1;

@Controller
public final class ControllerV1 {

  @RequestMapping(value="^\\/v1\\/user\\/(?<userId>\\d+)\\.json$", methods={GET})
  public final String fooV1(@Path("userId") final String userId) {
    return "v1: " + userId;
  }

}

And another for v2:

package com.foo.api.controllers.v2;

@Controller
public final class ControllerV2 {

  @RequestMapping(value="^\\/v2\\/user\\/(?<userId>\\d+)\\.json$", methods={GET})
  public final String fooV2(@Path("userId") final String userId) {
    return "v2: " + userId;
  }

}

Note clean separation using a unique package declaration.

Accept Header

Another, slightly more RESTful approach, is using the Accept HTTP request header to identify the desired version of a resource. This is somewhat analogous to client/server “content negotiation”.

In the interest of brevity, I won’t write a complete implementation here. However, a key takeaway is that you can use Curacao’s @Header annotation to extract the value of any request header. From there, your business logic in the controller can examine the header value to make a decision about what API version is invoked.

@Controller
public final class HeaderController {

  @RequestMapping(value="^\\/foo", methods={GET})
  public final String headerDemo(@Header("Accept") final String accept) {
    if (accept.contains("v2")) {
      // V2
    } else {
      // V1
    }
  }  

}

In addition to @Header, there are a number of “convenience” request header annotations you can use to decorate your controller method arguments:

  • @Accept — convenience for the Accept request header
  • @UserAgent — convenience for the User-Agent request header
  • @ContentType — convenience for the Content-Type request header
  • @Authorization — convenience for the Authorization request header
  • … and of course, many more in the com.kolich.curacao.annotations.parameters.convenience package.

Part 2

Next in this series, Writing Versioned Service APIs With Curacao: Part 2 discusses routing strategies with Curacao.

Enjoy!