Lazada Product Feeds to MongoDB

Jadi denger-denger kemarin pada rame bikin affiliasi Lazada. Sepertinya akan rame ini.. seperti affiliasi situs e-commerce terbesar di dunia kemarin :)

Iseng-iseng bikin juga, dengan import product feeds dari lazada.co.id. (Tips ini juga bisa berlaku untuk product feeds affiliasi lain).

Database yang dipakai adalah mongodb. Cuma butuh 1 baris command.

whe:lazada$ mongoimport -d lazada -c produk --type csv --headerline --file ID.csv
connected to: 127.0.0.1
Sat Aug 31 18:51:56.777 check 9 14093
Sat Aug 31 18:51:56.777 imported 14092 objects
whe:lazada$ mongo
MongoDB shell version: 2.4.4
connecting to: test
> use lazada
switched to db lazada
> db.produk.find().forEach(printjson);
{
	"_id" : ObjectId("5221d8da638398383c4521c1"),
	"simple_sku" : "TI205FAABBQAANID-67247",
	"parent_sku" : "TI205FAABBQAANID",
	"availability_instock" : "true",
	"brand" : "Timberland",
	"product_name" : "Timberland TBL13319JS/02B",
	"description" : "Desain sportyUnisexStainless steelGaransi 2 tahun",
	"currency" : "IDR",
	"price" : 2750000,
	"discount" : 825000,
	"discounted_price" : 1925000,
	"discount_percentage" : 30,
	"cat_m" : "MEGA DEAL",
	"parent_of_cat_1" : "Time to Get Watch",
	"category_1" : "",
	"parent_of_parent_of_cat2" : "",
	"parent_of_cat_2" : "",
	"category_2" : "",
	"parent_of_parent_of_cat3" : "",
	"parent_of_cat3" : "",
	"category_3" : "",
	"image_url_2" : "http://srv-live.lazada.co.id/p/image-85816-2-product.jpg",
	"image_url_3" : "",
	"image_url_4" : "",
	"image_url_5" : "",
	"picture_url" : "http://srv-live.lazada.co.id/p/image-85816-1-product.jpg",
	"deeplink" : "www.lazada.co.id/Timberland-TBL13319JS02B-61858.html#Q0C10ynu0626",
	"delivery_period" : "1 - 4 days"
}
{
	"_id" : ObjectId("5221d8da638398383c4521c2"),
	"simple_sku" : "TI205FAABBQCANID-67249",
	"parent_sku" : "TI205FAABBQCANID",
	"availability_instock" : "true",
	"brand" : "Timberland",
	"product_name" : "Timberland TBL13319JSTB/02",
	"description" : "Desain sportyUnisexStainless steelGaransi 2 tahun",
	"currency" : "IDR",
	"price" : 2750000,
	"discount" : 825000,
	"discounted_price" : 1925000,
	"discount_percentage" : 30,
	"cat_m" : "MEGA DEAL",
	"parent_of_cat_1" : "Time to Get Watch",
	"category_1" : "",
	"parent_of_parent_of_cat2" : "",
	"parent_of_cat_2" : "",
	"category_2" : "",
	"parent_of_parent_of_cat3" : "",
	"parent_of_cat3" : "",
	"category_3" : "",
	"image_url_2" : "http://srv-live.lazada.co.id/p/image-06816-2-product.jpg",
	"image_url_3" : "",
	"image_url_4" : "",
	"image_url_5" : "",
	"picture_url" : "http://srv-live.lazada.co.id/p/image-06816-1-product.jpg",
	"deeplink" : "www.lazada.co.id/Timberland-TBL13319JSTB02-61860.html#Q0C6x08wRy",
	"delivery_period" : "1 - 4 days"
}
.......

Sekali masuk database, selanjutnya terserah kreatifitas Anda dalam membikin situs :)

Using Mega.co.nz as Your VPS Backup

Lost all host data is a very bad experience. Also paying your host provider to get your own backup is not fun anymore. And backup services usually have limited feature to send our backup. Until mega.co.nz launched.

50Gb space for free account is very big. It was impossible for me to upload that amount of files. I was thinking to make it a backup storage for my VPS(ses).

So here’s the plan: VPS –> daily/weekly backups –> mega API (megous) –> mega.co.nz
Using megous as our tool, you can install it from github or it’s official page. We will use Ubuntu’s ppa to do this.

#adding add-apt-repository binary
sudo apt-get install python-software-properties software-properties-common
sudo add-apt-repository ppa:megous/ppa
sudo apt-get update

We need to add glib-networking to run megatools

sudo apt-get install glib-networking
sudo apt-get install megatools

You can check your account on Mega.co.nz

megadf -u yourmail@foo.com -p password -h
Total: 50.0 GiB
Used: 25.7 GiB
Free: 24.3 GiB

This is just an example for my simple need: just to backup all www files and it’s database. Nothing more.

#!/bin/bash
TAR=`which tar`
MEGASYNC=`which megasync`
MV=`which mv`
NOHUP=`which nohup`
#dailybackup
$TAR -cvjpf /home/user/backup/daily/backup_daily.tar.bz2 /var/www/yourbelovedsite.com
$NOHUP $MEGASYNC -u yourmail@foo.com -p password --local /home/user/backup/daily/ --remote /Root/daily

For database backup script, we can use this script and point to the same directory with previous file backup. Don’t forget to execute it with cron.

crontab -e
#add cron
@daily /path/your/script.sh

That’s it :)

show resource usage by each user

Sometimes when my box suddenly got traffic spike or some user hogging resources up, plus.. I was so lazy to investigate the problem, this ps command helps so much:

ps -eo user,pcpu,pmem | tail -n +2 | awk '{
        num[$1]++; 
        cpu[$1] += $2; 
        mem[$1] += $3} END{
        printf("NPROC\tUSER\tCPU\tMEM\n"); 
        for (user in cpu) printf("%d\t%s\t%.2f%\t%.2f%\n",num[user], user, cpu[user], mem[user]) 
}'

Output:

NPROC        USER        CPU        MEM
5        user1        59.80%        1.20%
1        user2        0.50%        0.00%
3        user3        0.10%        0.00%
1        user4        5.40%        0.10%
1        user5        7.60%        0.20%
2        user6        0.00%        0.00%
404        root        6.50%        2.30%
.
.
.