Day 24 Big Data on Hadoop Udemy

Day 24 Big Data on Hadoop Udemy | Hive

We got our image started in the virtual sand box. We needed to disable hyper-v as Windows 10 doesn't allow Virtual Box Hyper-V and Hyper-V on the windows 10.

We downloaded the dataset , now uploading the dataset in Hive

Ambari Sandbox allows us to directly use Hive from the url.

we can import data in the csv format and query data as sql relational although it is not stored relational. we can also visualize the selected data columns.
https://www.techrepublic.com/blog/10-things/10-things-you-should-know-about-hyper-v/
https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/about/

https://stackoverflow.com/questions/11183572/whats-the-difference-between-rank-and-dense-rank-functions-in-oracle
Difference between RANK , DENSE RANK and ROW_NUMBER
ROW_NUMBER will always give unique
RANK will give same rank for the first 4 same but 5th rank for the 5th .
DENSE_RANK will give first 4 same but 2 for the 5th like a group by.

Scrolling Page Title
http://www.techtricky.com/javascript-code-to-scrolling-page-title/

Comments

Day 15 GCP Recommendations , Cloud SQL PySpark DataProc

Collaborative Filtering - RDD-based API https://spark.apache.org/docs/latest/mllib-collaborative-filtering.html PySpark https://spark.apache.org/docs/0.9.0/python-programming-guide.html Managed Hadoop & Spark https://cloud.google.com/dataproc/ Fully-Managed PostgreSQL BETA & MySQL https://cloud.google.com/sql/ Cloud sql can run 1 petabit per second

Day 18 GCP DataLab, Big Query from Client Side , Pandas- Python

'sudo su -' vs 'sudo -i' vs 'sudo /bin/bash' - when does it matter which is used, or does it matter at all? https://askubuntu.com/questions/376199/sudo-su-vs-sudo-i-vs-sudo-bin-bash-when-does-it-matter-which-is-used docker ps will show only running containers by default. To see all containers: docker ps -a https://docs.docker.com/v1.11/engine/reference/commandline/ps/ https://8081-dot-2337103-dot-devshell.appspot.com/tree/datalab root1234 - paraphrase DataLab gives the ability to share a notebook with other people , at the same time use the cloud for computing n storage. https://cloud.google.com/bigquery/docs/reference/libraries#client-libraries-usage-python https://github.com/google/google-api-javascript-client http://stackoverflow.com/questions/12479895/obtaining-bigquery-data-from-javascript-code Python Data Analysis Library http://pandas.pydata.org/

Day 13 - Microsoft artificial Intelligence | Arduino Robotic Kit

Using Visual Studio for python -- back to debugging mode , feels amazing. So finally after managing to get 4 18650 batteries , 2 of them are discharged , now not sure how to recharge them. Good Resources for AI https://blog.goodaudience.com/learn-ai-for-free-5b186cde3990 Python Math Operators https://www.digitalocean.com/community/tutorials/how-to-do-math-in-python-3-with-operators Reading an Input form cmd line with python https://stackoverflow.com/questions/70797/user-input-and-command-line-arguments Convert string to int in Python https://guide.freecodecamp.org/python/how-to-convert-strings-into-integers-in-python/ https://www.programiz.com/python-programming/if-elif-else

Day 20 Google APIs, Google Application Default Credentials

Searching for objects attribute value, it has to be Datastore . Remember that BigTable, you can only search by key. High-throughput writes of wide-column data. Well, that is BigTable , right, because it's supporting high-throughput writes. Warehousing structured data. So what's the data warehouse technology on Google Cloud? That's, which one, BigQuery . To create and test new machine learning methods. Well, if you're writing new machine learning methods, then TensorFlow . Develop Big Data algorithms interactively in Python.Well, interactive development in Python is done best with Datalab . Well, interactive No-ops, custom machine learning applications at scale. No-ops ML at scale, then that's a role for Cloud ML. Automatically reject inappropriate image content. Rejecting image content where it is inappropriate. Well, that could be the Vision API. So you could use a Vision API to basically see if this is safe content or not safe...

Day 8 - Microsoft AI | Arduino Robotic Kit

Arduino Robotic Kit https://photos.app.goo.gl/Ng6MmaRnxHHWUHpJ6 Just got my Arduino robotic kit , quite excited :-) , Let us start exploring the various components. HC SR 04 Ultrasonic Ranging Module https://www.mouser.com/ds/2/813/HCSR04-1022824.pdf HC - SR04 provides 2cm - 400cm non-contact measurement function, the ranging accuracy can reach to 3mm Vcc , Trigger , Echo n Ground You only need to supply a short 10uS pulse to the trigger input to start the ranging, and then the module will send out an 8 cycle burst of ultrasound at 40 kHz and raise its echo. The Echo is a distance object that is pulse width and the range in proportion https://randomnerdtutorials.com/complete-guide-for-ultrasonic-sensor-hc-sr04/ https://www.arduino.cc/en/Reference/Libraries https://www.youtube.com/watch?v=6F1B_N6LuKw

Day 7 Hands on with Git Repo and RStudio , R 101 BigDataUniversity

Removing a remote http://stackoverflow.com/questions/9224754/how-to-remove-origin-from-git-repository Kickstarting R - Writing R scripts https://cran.r-project.org/doc/contrib/Lemon-kickstart/kr_scrpt.html Source on Save https://support.rstudio.com/hc/en-us/articles/200484448-Editing-and-Executing-Code Ctrl+L — Clear the Console https://support.rstudio.com/hc/en-us/articles/200404846-Working-in-the-Console User Defined Functions in R http://www.statmethods.net/management/userfunctions.html Issue pushing new code in Github http://stackoverflow.com/questions/20939648/issue-pushing-new-code-in-github Git refusing to merge unrelated histories http://stackoverflow.com/questions/37937984/git-refusing-to-merge-unrelated-histories

Day 5 IBM Watson , DataCamp Purchase, IT Pros Attack n RedMonk rankings

IBM Watson http://www.jenunderwood.com/2017/03/28/ibm-watson-cognitive-computing/?utm_content=bufferb4ded&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer The DataCamp Intro to R is free but later courses are paid. Will try complete Coursera first and get some heads on and then I can buy DataCamp courses as well. https://www.tripwire.com/state-of-security/featured/90-pros-expect-attacks-risk-vulnerability-iiot-2017/ http://redmonk.com/sogrady/2017/03/17/language-rankings-1-17/

Day 14 Swirl

install.packages("swirl") https://github.com/swirldev/swirl_courses#swirl-courses https://en.wikipedia.org/wiki/YAML http://yaml.org/ | You can exit swirl and return to the R prompt (>) at any time by pressing the Esc key. If you are | already at the prompt, type bye() to exit and save your progress. When you exit properly, you'll see a | short message letting you know you've done so. | When you are at the R prompt (>): | -- Typing skip() allows you to skip the current question. | -- Typing play() lets you experiment with R on your own; swirl will ignore what you do... | -- UNTIL you type nxt() which will regain swirl's attention. | -- Typing bye() causes swirl to exit. Your progress will be saved. | -- Typing main() returns you to swirl's main menu. | -- Typing info() displays these options again. | Let's get started! sqrt() function and to take the absolute value, use the abs() function Vector of unequal length Artihmetic Op...

Moment to Ignite

I know what you're thinking.It's nothing like that , it's infact the good ol friendship stuff. If i write explicitly in the title no one would bother to read it. I remember my hindi sir telling it takes 2 hours to write something and 2 days to give an apt title. In this modern era there is only one thing that interests everyone controversies, gossip ,all the masala put in together.Anyways I'm deviating a lot. How do you make friends? I might have asked this earlier...take it as a refresher... Yeah yeah i know i know....you cant make friends, you become friends.But again how?? Most of you will be like, whose bothered how now that we are friends.Put aside similar interests,likings , all the rational logical astute observations.Could it be some kind of destiny thing.It hardly sounds like me but then i have changed over time,I have started keeping my views open to every single possibility. Let me put a more real situation, what if you knew each other all the way a...

Helplessness

So we had no project and to add the to misery no jobs.We were told by our placement dept tht our dept so called MEDICAL ELECTRONICS wasn't allowed by most of the companies. We all wer dejected as all of us had worked hard and wanted to get some job.I tht why sit n crib all time when u have so much time to enjoy. So i hit wit my family to Rajasthan.It was a nice but extremely hectic trip for me. We went by flight to jaipur then next day jodhpur by bus the same day udaipur by bus ,whr i came to know tht companies wer allowing us n the placement dept scammed us. So next day we left frm udaipur to mumbai by train n then the same day to bangalore by bus. In 5 days i had spanned across 5 states. I rechd bangalore and had accenture next day so went thro the placement bible RS Aggarwal. Sadly i didn't clear then i sat for HCL which again i didn't clear so i was amidst doubt within myself. But still i didn't lose hope n worked harder.

The World's a Stage n we Actors......

Search This Blog