Compile and install Thrift 0.8.0 on CentOS 5.8 and 6
When you want to install Thrift on CentOS 5.8 or 6, it is better to download the tallbar from Apache thrift website. Don't checkout the source code from SVN or git because CenOS 5.8 and 6 don't have...
View ArticleHive Server 2 in CDH4.1
Just gave a try Hive server 2 in CDH 4.1 and encounter couple of issues: Hive server 2 support LDAP and Kerberos, but only supports Simple bind. Unfortunately our LDAP server only supports SASL. If you...
View ArticleSetup Tomcat on CentOS for Windows Authentication using SPNEGO
I setup a Tomcat server running on a Linux box with SPNEGO, so that the users can Single-Sign-On the server without typing their password. You can follow the instructions on...
View ArticleDB2 Client Install on Linux
db2ls can show the current installation[bewang@logs]$ db2lsInstall Path Level Fix Pack Special Install Number Install Date Installer UID...
View ArticleImport users to Cloudera Hue 2 from Hue 1.2.0
I thought it should not be too hard to import users from Hue 1.2.0 into Hue 2 (CDH 4.1) because this page doesn't mention special steps:...
View ArticleHue 2.0 failed when syncdb with a MySQL database
If you use MySQL as HUE 2.0 database, and the default charset of the database is UTF8, you will have trouble to run syncdb because the migration...
View ArticleTips of Jenkins and Nexus
Fingerprint Issue in Jenkins: delete the dir d:\.jenkins\fingerprintsWaiting for Jenkins to finish collecting dataERROR: Asynchronous execution failurejava.util.concurrent.ExecutionException:...
View ArticleStart Puppet Master As Non-Root User
My company has a lengthy process if we need root permission to deploy something into a Linux server, 3-4 days for approval. Sound impossible? Unfortunately it is true. And I don't want to run my puppet...
View ArticleUsing Puppet to Manage Java Application Deployment
Deployment of a Java application can be done in a different ways. We can do Build one jar with all exploded dependent jars and your application. You don't have to worry about your classpath.Build a...
View ArticleCDH4.1.0 Hive Debug Option Bug
Hive 0.9.0 supports remote debug.Run the following command, and Hive will suspend and listening on 8000.hive --debugBut there is a bug in Hive CDH4.1.0 which blocks you from using this option. You will...
View ArticleBad performance of Hive meta store for tables with large number of partitions
Just found this article Batch fetching - optimizing object graph loadingWe have some tables with 15K ~ 20K partitions. If I run a query scanning a lot of partitions, Hive could use more than 10 minutes...
View ArticleHive Metastore Configuration
Recently I wrote a post for Bad performance of Hive meta store for tables with large number of partitions. I did tests in our environment. Here is what I found:Don't configure a hive client to access...
View ArticlePuppet require vs. include vs. class
According to puppet reference include and require: Both include and require are functions;Both include and require will "Evaluate one or more classes";Both include and require cannot handle parametered...
View ArticleHave to read all from Ruby PTY output
I wrote a Ruby script to call ssh-copy-id to distribute my public key to a list of hosts. It took me a while to make it work. The code is simple: read the password, and call ssh-copy-id for each host...
View ArticleBash single quote escape in running a Hive query
If you want to run this hive query using SSH, you probably will be headache how to escape: select * from my_table where date_column = '2013-01-11';The correct command looks like this: ssh myserver...
View ArticleHBase LeaseException issue
Excerpt from http://hbase.apache.org/book.html. If you google "hbase LeaseException", this page may not be on the first page. 12.5.2. LeaseException when calling Scanner.nextIn some situations clients...
View ArticleScala SBT
Requirements: Include scala-library.jar in the zip file because the machine may not have scala installed. Include scripts and configure files into the zip just like Maven assembly. Include the jar in...
View ArticleGit-SVN
Here are some notes when I play with git-svn: Clone a project. You can run the following commands if the project follows the standard layout with trunk/tags/branches. root |- ProjectA | |- branches |...
View ArticleExtract compressed Tar
Extract GZIP compressed TAR filetar xzvf file.tar.gzExtract BZ2 compressed TAR filetar xvjpf file.tar.gz
View Article
