Description:
Practice your Python Pandas data science skills with problems on StrataScratch! https://www.stratascratch.com/?via=keith In this video we walk through web scraping in Python using the beautiful soup library. We start with a brief introduction to HTML & CSS and discuss what web scraping is. Next we start getting into the basics of the beautiful soup library. This includes how to load a webpage, the basic commands you need to know such as find & find_all, grabbing strings from an HTML elements, etc. The final section of this tutorial is a series of exercises where you can practice your skills. In this section we scrape a webpage for links, we learn how to scrape a table and load it into a pandas dataframe, and we see how you can scrape & download a web image. Hope you enjoy! I’m looking into making future videos on more complex things you can do with web scraping as well as other libraries that are helpful such as Selenium & ScraPy. Subscribe to not miss those. Join the Python Army to get access to perks! YouTube - https://www.youtube.com/channel/UCq6XkhO5SZ66N04IcPbqNcw/join Patreon - https://www.patreon.com/keithgalli --------------------- Resources used in this video Simple webpage: https://keithgalli.github.io/web-scraping/example.html Example webpage: https://keithgalli.github.io/web-scraping/webpage.html Link to source code: https://github.com/KeithGalli/web-scraping Beautiful Soup Documentation: https://www.crummy.com/software/BeautifulSoup/bs4/doc/ CSS Selector Reference: https://www.w3schools.com/cssref/css_selectors.php --------------------- Learn more about HTML/CSS @Traversy Media HTML Crash Course: https://www.youtube.com/watch?v=UB1O30fR-EE @Traversy Media CSS Crash Course: https://www.youtube.com/watch?v=yfoY53QXEnI Codecademy: https://www.codecademy.com/catalog/language/html-css --------------------- Video timeline! 0:00 - Intro & Video Overview 1:09 - What is web scraping? 3:51 - Introduction to HTML Using the beautiful soup library (5:29) 6:31 - Loading in a webpage (requests library) 8:21 - Starting to scrape 9:18 - find & find_all methods 16:00 - Finding specific text/strings in our HTML (regex) 18:38 - Select method (CSS path selections) 25:55 - Grabbing the string/text from an HTML element 28:17 - Getting a property of HTML element (href, src, id, class, etc) 29:41 - Code navigation (parents, children, siblings) Let’s practice our skills! (33:57) 35:53 - Exercise #1: Grab all social links on webpage in 3 different ways 42:09 - Exercise #2: Scrape an HTML table into a Pandas Dataframe 53:09 - Exercise #3: Grab all fun facts that contain the word “is” 57:59 - Exercise #4: Use beautiful soup to help download an image from a webpage 1:04:20 - Exercise #5: Solve the mystery challenge!!! --------------------- Follow me on social media! Instagram | https://www.facebook.com/unsupportedbrowser Twitter | https://twitter.com/keithgalli --------------------- If you are curious to learn how I make my tutorials, check out this video: https://www.youtube.com/watch?v=LEO4igyXbLs *I use affiliate links on the products that I recommend. I may earn a purchase commission or a referral bonus from the usage of these links.
Preparing download options
http://unidownloader.com/ website is the best way to download a video or a separate audio track if you want to do without installing programs and extensions.
The UDL Helper extension is a convenient button that is seamlessly integrated into YouTube, Instagram and OK.ru sites for fast content download.
UDL Client program (for Windows) is the most powerful solution that supports more than 900 websites, social networks and video hosting sites, as well as any video quality that is available in the source.
UDL Lite is a really convenient way to access a website from your mobile device. With its help, you can easily download videos directly to your smartphone.
The best quality formats are FullHD (1080p), 2K (1440p), 4K (2160p) and 8K (4320p). The higher the resolution of your screen, the higher the video quality should be. However, there are other factors to consider: download speed, amount of free space, and device performance during playback.
The browser/computer should not freeze completely! If this happens, please report it with a link to the video. Sometimes videos cannot be downloaded directly in a suitable format, so we have added the ability to convert the file to the desired format. In some cases, this process may actively use computer resources.
You can download a video to your smartphone using the website or the PWA application UDL Lite. It is also possible to send a download link via QR code using the UDL Helper extension.
The most convenient way is to use the UDL Client program, which supports converting video to MP3 format. In some cases, MP3 can also be downloaded through the UDL Helper extension.
This feature is available in the UDL Helper extension. Make sure that "Show the video snapshot button" is checked in the settings. A camera icon should appear in the lower right corner of the player to the left of the "Settings" icon. When you click on it, the current frame from the video will be saved to your computer in JPEG format.
It costs nothing. Our services are absolutely free for all users. There are no PRO subscriptions, no restrictions on the number or maximum length of downloaded videos.