Search engines have come a long way in understanding JavaScript, but issues with rendering and load times can still impact your crawl budget and prevent search engines from indexing valuable content! Finding the optimal solution that provides the best user experience, whilst also satisfying the bots can be a challenge. This talk will cover the differences between these solutions, a number of tools and metrics you can use, and other significant considerations to take into account when proposing a rendering solution to your developers.

1. hey Googlebot,
did you cache that?
Petra Kis-Herczegh
@chameleon_jrnl | @botify | #brightonSEO
how to help bots discover valuable content

2. @chameleon_jrnl | @botify | #brightonSEO
Hello Googlebot, welcome to my website

3. @chameleon_jrnl | @botify | #brightonSEO
HTML,
Stylesheets, DOM,
JSON, JS, API
...here are 241 resources per page for you to render

4. very important
internal
linking
reviews
@chameleon_jrnl | @botify | #brightonSEO
super awesome
content
more awesome
content
pagination
third party
JavaScript
more
pagination
awesome
content
blocked by
robots.txt
duplicate
content
lazy loading
uncrawlable
links
HTML,
Stylesheets, DOM,
JSON, JS, API

5. @chameleon_jrnl | @botify | #brightonSEO
5.2 Billion Webpages

6. @chameleon_jrnl | @botify | #brightonSEO
Google’s resources are finite

7. @chameleon_jrnl | @botify | #brightonSEO
source: botify.com/blog/do-slow-page-load-times-negatively-impact-how-google-crawls-your-site
For large websites, load times are definitely impacting Google’s crawl

8. “There’s so many different factors [that cause pages to load slowly]. Sometimes it’s
your servers, or sometimes your servers respond really quickly but there’s a ton
of JavaScript that has to be processed first. JavaScript is a very expensive
resource because it has to be fully downloaded, parsed, and then executed.”
source: https://www.youtube.com/watch?v=XUOD6pcvnso

9. How long
does Google
take to fetch,
render and
cache
your pages...
source: https://developers.google.com/search/docs/guides/javascript-seo-basics

10. ...and how
long until
your pages
make it to
Google’s
index?
source: https://developers.google.com/search/docs/guides/javascript-seo-basics

11. @chameleon_jrnl | @botify | #brightonSEO
“Googlebot caches aggressively in order to reduce
network requests and resource usage “
source: https://developers.google.com/search/docs/guides/javascript-seo-basics

12. @chameleon_jrnl | @botify | #brightonSEO
“...if we have something that is already cached, that
doesn’t count towards your crawl budget.”
source: https://www.youtube.com/watch?v=EZtCgrpa6ss

13. GSC Crawl Stats
Average response time
doesn’t include executing
scripts, images, etc.
Google Search Console >> Settings >> Crawl Stats

14. GSC Crawl Stats
Google Search Console >> Settings >> Crawl Stats
Doesn’t include information
about resources loaded from
different domains, such as
images from your CDN or any
third party scripts.

15. @chameleon_jrnl | @botify | #brightonSEO
GSC Crawl Stats
Google Search Console >> Settings >> Crawl Stats
Doesn’t include information
about resources loaded from
different domains, such as
images from your CDN or any
third party scripts.

16. @chameleon_jrnl | @botify | #brightonSEO
GSC Crawl Stats
Total download bytes include
HTML, associated images,
scripts, CSS for the associated
domain only. Cached
resources aren’t counted
Google Search Console >> Settings >> Crawl Stats

17. very important
internal
linking
reviews
@chameleon_jrnl | @botify | #brightonSEO
super awesome
content
more awesome
content
pagination
third party
JavaScript
more
pagination
awesome
content
blocked by
robots.txt
duplicate
content
lazy loading
uncrawlable
links
HTML,
Stylesheets, DOM,
JSON, JS, API

18. Understand
JavaScript
JavaScript elements
● Quick JavaScript Switcher
● Chrome DevTools (right click + inspect)
Talk to your developers!

19. Understand
JavaScript
JavaScript load time on different pages
and against competitors (for users)
● pagespeed.compare
● Lighthouse
● Page Speed Insights

20. Understand
JavaScript
Rendered Source (for bots)
● URL inspection tool (GSC)
● Mobile-friendly test
● View Rendered Source

21. Understand
JavaScript
Use JavaScript configured crawls!
● include resources important to
load content and links
● exclude tracking, analytics, ads

22. @chameleon_jrnl | @botify | #brightonSEO
Large & JavaScript heavy sites Every website
Different challenge

23. @chameleon_jrnl | @botify | #brightonSEO
Crawl, Render, Index SERP & Website Engagement
Different Journey

24. @chameleon_jrnl | @botify | #brightonSEO
Get pages Indexed Buy Services & Product
Different Goal

25. @chameleon_jrnl | @botify | #brightonSEO
Full SEO funnel
Crawl & Render Index Rank Click Convert

26. @chameleon_jrnl | @botify | #brightonSEO

27. @chameleon_jrnl | @botify | #brightonSEO
Server Response
Total Rendering Time
Core Web Vitals
Site Speed

28. @chameleon_jrnl | @botify | #brightonSEO

29. @chameleon_jrnl | @botify | #brightonSEO
Crawl Web Vitals

30. Server Response time Download, parse, execute time Page Load
+ =

31. Server Response time Download, parse, execute time Page Load
+ =
Client Side Rendering (CSR)
User
&
Bot
request
Server
response
Page fully
loaded
Initial HTML + JavaScript
No additional
implementation required
+
Users CPU will impact
load time
Potential rendering issues
for bots
-

32. Server Response time Download, parse, execute time Page Load
+ =
Server Side Rendering (SSR)
User
&
Bot
request
Page fully
loaded
Dynamic HTML (+ JS)
Server
response
Initial HTML + JS
Build time
Improved load time for
users and bots
Can provide a solution to
rendering challenges
+
Expensive
Time to First Byte can be
delayed
Complex
Third-party JS might still
not render
-

33. Server Response time Download, parse, execute time Page Load
+ =
Prerendering or Static Site Generation
User
&
Bot
request
Page fully
loaded
Static HTML (+ JS rehydration)
Server
response
Build time
Prerendered
Cache
Static Site
Generation
Build time doesn’t impact
Time to First Byte
No rendering challenges
for bots
Users and bots are
served fast Static HTML
+
Expensive to scale
Interactive content
requires rehydration in
browser for users
-

34. @chameleon_jrnl | @botify | #brightonSEO

35. @chameleon_jrnl | @botify | #brightonSEO
“cloaking!”
“black hat!”
“penalties”
IT’S
NOT

36. @chameleon_jrnl | @botify | #brightonSEO
Dynamic Rendering
source:https://developers.google.com/search/docs/guides/dynamic-rendering

37. “Dynamic Rendering is not cloaking”
source:https://developers.google.com/search/docs/guides/dynamic-rendering#cloaking
see also: https://blogs.bing.com/webmaster/october-2018/bingbot-Series-JavaScript,-Dynamic-Rendering,-and-Cloaking-Oh-My

38. @chameleon_jrnl | @botify | #brightonSEO

39. @chameleon_jrnl | @botify | #brightonSEO
Dynamic Rendering
Deliver the same content seen by the user
Avoid rendering issues caused by JavaScript
(empty pages, missing or uncrawlable links,
missing third party content)
Make crawling more efficient by
delivering the content faster
Change the content
Deceive the crawler

40. Server Response time Download, parse, execute time Page Load
+ =
Dynamic Rendering
Page fully
loaded
Static HTML
Server
response
Build time
Prerendering
Cache
Page fully
loaded
Initial HTML + JavaScript
Server
response
User
&
Bot
request
User
Bot

41. Server Response time Download, parse, execute time Page Load
+ =
Hybrid Dynamic Rendering
Page fully
loaded
Static HTML
Server
response
Prerendering
Cache
Page fully
loaded
Initial HTML + JavaScript
Server
response
User
&
Bot
request
User
Bot
API fetch
Build time

42. very important
internal
linking
reviews
@chameleon_jrnl | @botify | #brightonSEO
super awesome
content
more awesome
content
pagination
third party
JavaScript
more
pagination
awesome
content
blocked by
robots.txt
duplicate
content
lazy loading
uncrawlable
links
HTML,
Stylesheets, DOM,
JSON, JS, API

43. @chameleon_jrnl | @botify | #brightonSEO
Hello Googlebot, welcome to my website

44. @chameleon_jrnl | @botify | #brightonSEO
...all resources have been prerendered for you
Static HTML

45. more
pagination
reviews
@chameleon_jrnl | @botify | #brightonSEO
super awesome
content
more
awesome
content
pagination
awesome
content
blocked by
robots.txt
duplicate
content
crawlable links
and navigation
Static HTML
very important
internal
linking

46. @chameleon_jrnl | @botify | #brightonSEO

47. @chameleon_jrnl | @botify | #brightonSEO
It’s time to make a decision

48. Site needs to be
interactive for Users
Site has over
500,000 pages
SSG
Site has often changing
content and/or elements
CSR with Prerendering or
SSR or SSG with rehydration
No
Yes
No
No
Yes
Yes
Bots: Dynamic Rendering
Bots: Hybrid Dynamic Rendering
Users: combination of SSR, CSR and Prerendering
@chameleon_jrnl | @botify | #brightonSEO

49. @chameleon_jrnl | @botify | #brightonSEO
Is it worth it?

50. @chameleon_jrnl | @botify | #brightonSEO
March 2020

51. @chameleon_jrnl | @botify | #brightonSEO
Implemented Dynamic Rendering

52. @chameleon_jrnl | @botify | #brightonSEO
March 2021 vs March 2020

53. @chameleon_jrnl | @botify | #brightonSEO
March 2021 vs March 2020

54. 54
@chameleon_jrnl | @botify | #brightonSEO
March 2021 vs March 2020

55. @chameleon_jrnl | @botify | #brightonSEO
March 2021 vs March 2020
38%
Number of Pages
with Impressions
24%
Number of Pages
with Traffic
19%
Overall
Organic Traffic

56. @chameleon_jrnl | @botify | #brightonSEO
Conclusion: Bot and User experience go hand in hand...