Hakuna MapData! » shell

A user having surprising troubles running more resource-intensive Hive queries

| Posted in Troubleshooting |


The problem

A couple of months ago, one of our data analysts pernamently run into troubles when he wanted to run more resource-intensive Hive queries. Surprisingly, his queries were valid, syntactically-correct and run successfully on small data, but they just failed on larger datasets. On the other hand, other users were able to run the same queries successfully on the same large datasets. Obviously, it sounds like some permissions problem, however the user had right HDFS and Hive permissions.

What I did when assigning META region in HBase cluster took ages…

| Posted in Troubleshooting |


Today, I faced the issue of inability to assign META region to a regionserver when starting HBase cluster. Basically ROOT region was assigned correctly and quickly (just a matter of seconds), where META region was permanently in transition state. Obviously and unfortunately, it resulted in HBase cluster inaccessibility.

Since it have never encountered such situation before, I did not know best-to-use solution.

Recreating HBase table without violating region starting keys

| Posted in Doubts Resolving, Troubleshooting |


Today, I was using completebulkload to load larger amount of data into HBase. To keep this process well-balanced over the entire cluster (and thus faster), I was loading data into a table with pre-created regions. Since my bulkloading process failed in the middle a couple of times due to some misconfiguration, I needed to truncate the table each time.

I have noticed that a command like

hbase(main):017:0> truncate 'table'

will disable, drop and recreate the table with the same name settings (number of column familiers, compression, ttl, blocksize etc), but it does not maintain the region boundaries.

A couple of basic, but useful tricks when working with Apache HBase shell

| Posted in Programming |


I would like to share some basic tricks to use with Apache HBase shell that I have learned by reading HBase: The Definitive Guide by L.George, HBase in Action by N.Dimiduk and A.Khurana and taking part in Cloudera Training for Apache HBase.

In this post, I will create HBase table, populate it with sample data and scan it. Each step will demonstrate a different technique to achieve the goal.

Create the User table

You can pipe commands to the hbase shell and easily create the table using one single command (without the need to explicitely launch the HBase shell first).

$ echo "create 'user', 'info'" | hbase shell
create 'user', 'info'
0 row(s) in 1.7610 seconds