Some helpful Tips

  • 0

Some helpful Tips

1. How-to-run-a-hive-query-using-yesterdays-date

Use from_unixtime(unix_timestamp()-1*60*60*24, ‘yyyy-MM-dd’); in your hive query.

For example:

select * from sample where date1=from_unixtime(unix_timestamp()-1*60*60*24, ‘yyyy-MM-dd’);

2. How to diff file(s) in HDFS

How to diff a file in HDFS and a file in the local filesystem:
diff <(hadoop fs -cat /path/to/file) /path/to/localfile

How to diff two files in HDFS:
diff <(hadoop fs -cat /path/to/file1) <(hadoop fs -cat /path/to/file2)