From 4b9f7fcbe8351472867543357a9357fc7146349d Mon Sep 17 00:00:00 2001 From: Bryce Allen Date: Sat, 25 Dec 2021 00:55:38 -0500 Subject: [PATCH] seam carving --- Basic Julia syntax.jl | 263 +++++++ Lecture 1 - Images.jl | 807 ++++++++++++++++++++ Wonderful invention.jl | 256 +++++++ gradient.jl | 256 +++++++ hw0.jl | 359 +++++++++ hw1.jl | 1640 ++++++++++++++++++++++++++++++++++++++++ hw2.jl | 1035 +++++++++++++++++++++++++ hw3.jl | 1318 ++++++++++++++++++++++++++++++++ seam_carving.jl | 553 ++++++++++++++ 9 files changed, 6487 insertions(+) create mode 100644 Basic Julia syntax.jl create mode 100644 Lecture 1 - Images.jl create mode 100644 Wonderful invention.jl create mode 100644 gradient.jl create mode 100644 hw0.jl create mode 100644 hw1.jl create mode 100644 hw2.jl create mode 100644 hw3.jl create mode 100644 seam_carving.jl diff --git a/Basic Julia syntax.jl b/Basic Julia syntax.jl new file mode 100644 index 0000000..622db11 --- /dev/null +++ b/Basic Julia syntax.jl @@ -0,0 +1,263 @@ +### A Pluto.jl notebook ### +# v0.11.12 + +using Markdown +using InteractiveUtils + +# ╔═╡ 0d3aec92-edeb-11ea-3adb-cd0dc17cbdab +md"# A basic Julia syntax cheatsheet + +This notebook briefly summarizes some of the basic Julia syntax that we will need for the problem sets. +" + + +# ╔═╡ 3b038ee0-edeb-11ea-0977-97cc30d1c6ff +md"## Variables + +We can define a variable using `=` (assignment). Then we can use its value in other expressions: +" + +# ╔═╡ 3e8e0ea0-edeb-11ea-22e0-c58f7c2168ce +x = 3 + + +# ╔═╡ 59b66862-edeb-11ea-2d62-71dcc79dbfab +y = 2x + +# ╔═╡ 5e062a24-edeb-11ea-256a-d938f77d7815 +md"By default Julia displays the output of the last operation. (You can suppress the output by adding `;` (a semicolon) at the end.) +" + +# ╔═╡ 7e46f0e8-edeb-11ea-1092-4b5e8acd9ee0 +md"We can ask what type a variable has using `typeof`:" + +# ╔═╡ 8a695b86-edeb-11ea-08cc-17263bec09df +typeof(y) + +# ╔═╡ 8e2dd3be-edeb-11ea-0703-354fb31c12f5 +md"## Functions" + +# ╔═╡ 96b5a28c-edeb-11ea-11c0-597615962f54 +md"We can use a short-form, one-line function definition for simple functions:" + +# ╔═╡ a7453572-edeb-11ea-1e27-9f710fd856a6 +f(x) = 2 + x + +# ╔═╡ b341db4e-edeb-11ea-078b-b71ac00089d7 +md"Typing the function's name gives information about the function. To call it we must use parentheses:" + +# ╔═╡ 23f9afd4-eded-11ea-202a-9f0f1f91e5ad +f + +# ╔═╡ cc1f6872-edeb-11ea-33e9-6976fd9b107a +f(10) + +# ╔═╡ ce9667c2-edeb-11ea-2665-d789032abd11 +md"For longer functions we use the following syntax with the `function` keyword and `end`:" + +# ╔═╡ d73d3400-edeb-11ea-2dea-95e8c4a6563b +function g(x, y) + z = x + y + return z^2 +end + +# ╔═╡ e04ccf10-edeb-11ea-36d1-d11969e4b2f2 +g(1, 2) + +# ╔═╡ e297c5cc-edeb-11ea-3bdd-090f415685ab +md"## For loops" + +# ╔═╡ ec751446-edeb-11ea-31ba-2372e7c71b42 +md"Use `for` to loop through a pre-determined set of values:" + +# ╔═╡ fe3fa290-edeb-11ea-121e-7114e5c573c1 +let s = 0 + + for i in 1:10 + s += i # Equivalent to s = s + i + end + + s +end + +# ╔═╡ 394b0ec8-eded-11ea-31fb-27392068ef8f +md"Here, `1:10` is a **range** representing the numbers from 1 to 10:" + +# ╔═╡ 4dc00908-eded-11ea-25c5-0f7b2b7e18f9 +typeof(1:10) + +# ╔═╡ 6c44abb4-edec-11ea-16bd-557800b5f9d2 +md"Above we used a `let` block to define a new local variable `s`. +But blocks of code like this are usually better inside functions, so that they can be reused. For example, we could rewrite the above as follows: +" + +# ╔═╡ 683af3e2-eded-11ea-25a5-0d90bf099d98 +function mysum(n) + s = 0 + + for i in 1:n + s += i + end + + return s +end + +# ╔═╡ 76764ea2-eded-11ea-1aa6-296f3421de1c +mysum(100) + +# ╔═╡ 93a231f4-edec-11ea-3b39-299b3be2da78 +md"## Conditionals: `if`" + +# ╔═╡ 82e63a24-eded-11ea-3887-15d6bfabea4b +md"We can evaluate whether a condition is true or not by simply writing the condition:" + +# ╔═╡ 9b339b2a-eded-11ea-10d7-8fc9a907c892 +a = 3 + +# ╔═╡ 9535eb40-eded-11ea-1651-e33c9c23dbfb +a < 5 + +# ╔═╡ a16299a2-eded-11ea-2b56-93eb7a1010a7 +md"We see that conditions have a Boolean (`true` or `false`) value. + +We can then use `if` to control what we do based on that value:" + +# ╔═╡ bc6b124e-eded-11ea-0290-b3760cb81024 +if a < 5 + "small" + +else + "big" + +end + +# ╔═╡ cfb21014-eded-11ea-1261-3bc30952a88e +md"""Note that the `if` also returns the last value that was evaluated, in this case the string `"small"` or `"big"`, Since Pluto is reactive, changing the definition of `a` above will automatically cause this to be reevaluated!""" + +# ╔═╡ ffee7d80-eded-11ea-26b1-1331df204c67 +md"## Arrays" + +# ╔═╡ cae4137e-edee-11ea-14af-59a32227de1b +md"### 1D arrays (`Vector`s)" + +# ╔═╡ 714f4fca-edee-11ea-3410-c9ab8825d836 +md"We can make a `Vector` (1-dimensional, or 1D array) using square brackets:" + +# ╔═╡ 82cc2a0e-edee-11ea-11b7-fbaa5ad7b556 +v = [1, 2, 3] + +# ╔═╡ 85916c18-edee-11ea-0738-5f5d78875b86 +typeof(v) + +# ╔═╡ 881b7d0c-edee-11ea-0b4a-4bd7d5be2c77 +md"The `1` in the type shows that this is a 1D array. + +We access elements also using square brackets:" + +# ╔═╡ a298e8ae-edee-11ea-3613-0dd4bae70c26 +v[2] + +# ╔═╡ a5ebddd6-edee-11ea-2234-55453ea59c5a +v[2] = 10 + +# ╔═╡ a9b48e54-edee-11ea-1333-a96181de0185 +md"Note that Pluto does not automatically update cells when you modify elements of an array, but the value does change." + +# ╔═╡ 68c4ead2-edef-11ea-124a-03c2d7dd6a1b +md"A nice way to create `Vector`s following a certain pattern is to use an **array comprehension**:" + +# ╔═╡ 84129294-edef-11ea-0c77-ffa2b9592a26 +v2 = [i^2 for i in 1:10] + +# ╔═╡ d364fa16-edee-11ea-2050-0f6cb70e1bcf +md"## 2D arrays (matrices)" + +# ╔═╡ db99ae9a-edee-11ea-393e-9de420a545a1 +md"We can make small matrices (2D arrays) with square brackets too:" + +# ╔═╡ 04f175f2-edef-11ea-0882-712548ebb7a3 +M = [1 2 + 3 4] + +# ╔═╡ 0a8ac112-edef-11ea-1e99-cf7c7808c4f5 +typeof(M) + +# ╔═╡ 1295f48a-edef-11ea-22a5-61e8a2e1d005 +md"The `2` in the type confirms that this is a 2D array." + +# ╔═╡ 3e1fdaa8-edef-11ea-2f03-eb41b2b9ea0f +md"This won't work for larger matrices, though. For that we can use e.g." + +# ╔═╡ 48f3deca-edef-11ea-2c18-e7419c9030a0 +zeros(5, 5) + +# ╔═╡ a8f26af8-edef-11ea-2fc7-2b776f515aea +md"Note that `zeros` gives `Float64`s by default. We can also specify a type for the elements:" + +# ╔═╡ b595373e-edef-11ea-03e2-6599ef14af20 +zeros(Int, 4, 5) + +# ╔═╡ 4cb33c04-edef-11ea-2b35-1139c246c331 +md"We can then fill in the values we want by manipulating the elements, e.g. with a `for` loop." + +# ╔═╡ 54e47e9e-edef-11ea-2d75-b5f550902528 +md"A nice alternative syntax to create matrices following a certain pattern is an array comprehension with a *double* `for` loop:" + +# ╔═╡ 6348edce-edef-11ea-1ab4-019514eb414f +[i + j for i in 1:5, j in 1:6] + +# ╔═╡ Cell order: +# ╟─0d3aec92-edeb-11ea-3adb-cd0dc17cbdab +# ╟─3b038ee0-edeb-11ea-0977-97cc30d1c6ff +# ╠═3e8e0ea0-edeb-11ea-22e0-c58f7c2168ce +# ╠═59b66862-edeb-11ea-2d62-71dcc79dbfab +# ╟─5e062a24-edeb-11ea-256a-d938f77d7815 +# ╟─7e46f0e8-edeb-11ea-1092-4b5e8acd9ee0 +# ╠═8a695b86-edeb-11ea-08cc-17263bec09df +# ╟─8e2dd3be-edeb-11ea-0703-354fb31c12f5 +# ╟─96b5a28c-edeb-11ea-11c0-597615962f54 +# ╠═a7453572-edeb-11ea-1e27-9f710fd856a6 +# ╟─b341db4e-edeb-11ea-078b-b71ac00089d7 +# ╠═23f9afd4-eded-11ea-202a-9f0f1f91e5ad +# ╠═cc1f6872-edeb-11ea-33e9-6976fd9b107a +# ╟─ce9667c2-edeb-11ea-2665-d789032abd11 +# ╠═d73d3400-edeb-11ea-2dea-95e8c4a6563b +# ╠═e04ccf10-edeb-11ea-36d1-d11969e4b2f2 +# ╟─e297c5cc-edeb-11ea-3bdd-090f415685ab +# ╟─ec751446-edeb-11ea-31ba-2372e7c71b42 +# ╠═fe3fa290-edeb-11ea-121e-7114e5c573c1 +# ╟─394b0ec8-eded-11ea-31fb-27392068ef8f +# ╠═4dc00908-eded-11ea-25c5-0f7b2b7e18f9 +# ╟─6c44abb4-edec-11ea-16bd-557800b5f9d2 +# ╠═683af3e2-eded-11ea-25a5-0d90bf099d98 +# ╠═76764ea2-eded-11ea-1aa6-296f3421de1c +# ╟─93a231f4-edec-11ea-3b39-299b3be2da78 +# ╟─82e63a24-eded-11ea-3887-15d6bfabea4b +# ╠═9b339b2a-eded-11ea-10d7-8fc9a907c892 +# ╠═9535eb40-eded-11ea-1651-e33c9c23dbfb +# ╟─a16299a2-eded-11ea-2b56-93eb7a1010a7 +# ╠═bc6b124e-eded-11ea-0290-b3760cb81024 +# ╟─cfb21014-eded-11ea-1261-3bc30952a88e +# ╟─ffee7d80-eded-11ea-26b1-1331df204c67 +# ╟─cae4137e-edee-11ea-14af-59a32227de1b +# ╟─714f4fca-edee-11ea-3410-c9ab8825d836 +# ╠═82cc2a0e-edee-11ea-11b7-fbaa5ad7b556 +# ╠═85916c18-edee-11ea-0738-5f5d78875b86 +# ╟─881b7d0c-edee-11ea-0b4a-4bd7d5be2c77 +# ╠═a298e8ae-edee-11ea-3613-0dd4bae70c26 +# ╠═a5ebddd6-edee-11ea-2234-55453ea59c5a +# ╟─a9b48e54-edee-11ea-1333-a96181de0185 +# ╟─68c4ead2-edef-11ea-124a-03c2d7dd6a1b +# ╠═84129294-edef-11ea-0c77-ffa2b9592a26 +# ╟─d364fa16-edee-11ea-2050-0f6cb70e1bcf +# ╟─db99ae9a-edee-11ea-393e-9de420a545a1 +# ╠═04f175f2-edef-11ea-0882-712548ebb7a3 +# ╠═0a8ac112-edef-11ea-1e99-cf7c7808c4f5 +# ╟─1295f48a-edef-11ea-22a5-61e8a2e1d005 +# ╟─3e1fdaa8-edef-11ea-2f03-eb41b2b9ea0f +# ╠═48f3deca-edef-11ea-2c18-e7419c9030a0 +# ╟─a8f26af8-edef-11ea-2fc7-2b776f515aea +# ╠═b595373e-edef-11ea-03e2-6599ef14af20 +# ╟─4cb33c04-edef-11ea-2b35-1139c246c331 +# ╟─54e47e9e-edef-11ea-2d75-b5f550902528 +# ╠═6348edce-edef-11ea-1ab4-019514eb414f diff --git a/Lecture 1 - Images.jl b/Lecture 1 - Images.jl new file mode 100644 index 0000000..b1eab24 --- /dev/null +++ b/Lecture 1 - Images.jl @@ -0,0 +1,807 @@ +### A Pluto.jl notebook ### +# v0.11.12 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ 5e688928-e939-11ea-0e16-fbc80af390ab +using LinearAlgebra + +# ╔═╡ a50b5f48-e8d5-11ea-1f05-a3741b5d15ba +html"" + +# ╔═╡ 8a6fed4c-e94b-11ea-1113-d56f56fb293b +br = HTML("
") + +# ╔═╡ dc53f316-e8c8-11ea-150f-1374dbce114a +md"""# Welcome to 18.S191 -- Fall 2020! + +### Introduction to Computational Thinking for Real-World Problems""" + +# ╔═╡ c3f43d66-e94b-11ea-02bd-23cfeb878ff1 +br + +# ╔═╡ c6c77738-e94b-11ea-22f5-1dce3dbcc3ca +md"### " + +# ╔═╡ cf80793a-e94b-11ea-0120-f7913ae06f22 +br + +# ╔═╡ d1638d96-e94b-11ea-2ff4-910e399f864d +md"##### Alan Edelman, David P. Sanders, Grant Sanderson, James Schloss" + +# ╔═╡ 0117246a-e94c-11ea-1a76-c981ce8e725d +md"##### & Philip the Corgi" + +# ╔═╡ 27060098-e8c9-11ea-2fe0-03b39b1ddc32 +md"""# Class outline + +### Data and computation + +- Module 1: Analyzing images + +- Module 2: Particles and ray tracing + +- Module 3: Epidemic spread + +- Module 4: Climate change +""" + +# ╔═╡ 4fc58814-e94b-11ea-339b-cb714a63f9b6 +md"## Tools + +- Julia programming language: + +- Pluto notebook environment +" + +# ╔═╡ f067d3b8-e8c8-11ea-20cb-474709ffa99a +md"""# Module 1: Images""" + +# ╔═╡ 37c1d012-ebc9-11ea-2dfe-8b86bb78f283 +4 + 4 + +# ╔═╡ a0a97214-e8d2-11ea-0f46-0bfaf016ab6d +md"""## Data takes many forms + +- Time series: + - Number of infections per day + - Stock price each minute + - A piece for violin broadcast over the radio +$(HTML("
")) + +- Video: + - The view from a window of a self-driving car + - A hurricane monitoring station +$(HTML("
")) + +- Images: + - Diseased versus healthy tissue in a scan + - Deep space via the Hubble telescope + - Can your social media account recognise your friends? +""" + +# ╔═╡ 1697a756-e93d-11ea-0b6e-c9c78d527993 +md"## Capture your own image!" + +# ╔═╡ af28faca-ebb7-11ea-130d-0f94bf9bd836 + + +# ╔═╡ ee1d1596-e94a-11ea-0fb4-cd05f62471d3 +md"##" + +# ╔═╡ 8ab9a978-e8c9-11ea-2476-f1ef4ba1b619 +md"""## What is an image?""" + +# ╔═╡ 38c54bfc-e8cb-11ea-3d52-0f02452f8ba1 +md"Albrecht Dürer:" + +# ╔═╡ 983f8270-e8c9-11ea-29d2-adeccb5a7ffc +# md"# begin +# using Images + +# download("https://i.stack.imgur.com/QQL8X.jpg", "durer.jpg") + +# load("durer.jpg") +# end + +md"![](https://i.stack.imgur.com/QQL8X.jpg)" + +# ╔═╡ 2fcaef88-e8ca-11ea-23f7-29c48580f43c +md"""## + +An image is: + +- A 2D representation of a 3D world + +- An approximation + +""" + +# ╔═╡ 7636c4b0-e8d1-11ea-2051-757a850a9d30 +begin + image_text = + md""" + ## What *is* an image, though? + + - A grid of coloured squares called **pixels** + + - A colour for each pair $(i, j)$ of indices + + - A **discretization** + + """ + + image_text +end + +# ╔═╡ bca22176-e8ca-11ea-2004-ebeb103116b5 +md""" +## How can we store an image in the computer? + +- Is it a 1D array (`Vector`)? + +- A 2D array (`Matrix`)? + +- A 3D array (`tensor`)? +""" + +# ╔═╡ 0ad91f1e-e8d2-11ea-2c18-93f66c906a8b +md"""## If in doubt: Ask Julia! + +- Let's use the `Images.jl` package to load an image and see what we get +""" + +# ╔═╡ de373816-ec79-11ea-2772-ebdca52246ac +begin + import Pkg + Pkg.activate(mktempdir()) +end + +# ╔═╡ 552129ae-ebca-11ea-1fa1-3f9fa00a2601 +begin + Pkg.add(["Images", "ImageIO", "ImageMagick"]) + using Images +end + +# ╔═╡ fbe11200-e938-11ea-12e9-6125c1b56b25 +begin + Pkg.add("PlutoUI") + using PlutoUI +end + +# ╔═╡ 54c1ba3c-e8d2-11ea-3564-bdaca8563738 +# defines a variable called `url` +# whose value is a string (written inside `"`): + +url = "https://i.imgur.com/VGPeJ6s.jpg" + +# ╔═╡ 6e0fefb6-e8d4-11ea-1f9b-e7a3db40df39 +philip_file = download(url, "philip.jpg") # download to a local file + +# ╔═╡ 9c359212-ec79-11ea-2d7e-0124dad5f127 +philip = load(philip_file) + +# ╔═╡ 7703b032-ebca-11ea-3074-0b80a077078e +philip + +# ╔═╡ 7eff3522-ebca-11ea-1a65-59e66a4e72ab +typeof(philip) + +# ╔═╡ c9cd6c04-ebca-11ea-0990-5fa19ff7ed97 +RGBX(0.9, 0.1, 0.1) + +# ╔═╡ 0d873d9c-e93b-11ea-2425-1bd79677fb97 +md"##" + +# ╔═╡ 6b09354a-ebb9-11ea-2d5a-3b75c5ae7aa9 + + +# ╔═╡ 2d6c434e-e93b-11ea-2678-3b9db4975089 +md"##" + +# ╔═╡ 2b14e93e-e93b-11ea-25f1-5f565f80e778 +typeof(philip) + +# ╔═╡ 0bdc6058-e8d5-11ea-1889-3f706cea7a1f +md"""## + +- According to Julia / Pluto, the variable `philip` *is* an image + +- Julia always returns output + +- The output can be displayed in a "rich" way + +$(HTML("
")) + +- Arthur C. Clarke: + +> Any sufficiently advanced technology is indistinguishable from magic. +""" + +# ╔═╡ e61db924-ebca-11ea-2f79-f9f1c121b7f5 +size(philip) + +# ╔═╡ ef60fcc4-ebca-11ea-3f69-155afffe8ea8 +philip + +# ╔═╡ fac550ec-ebca-11ea-337a-dbc16848c617 +philip[1:1000, 1:400] + +# ╔═╡ 42aa8cfe-e8d5-11ea-3cb9-c365b98e7a8c +md" +## How big is Philip? + +- He's pretty big: +" + +# ╔═╡ 4eea5710-e8d5-11ea-3978-af66ee2a137e +size(philip) + +# ╔═╡ 57b3a0c2-e8d5-11ea-15aa-8da4549f849b +md"- Which number is which?" + +# ╔═╡ 03a7c0fc-ebba-11ea-1c71-79d750c97b16 +philip + +# ╔═╡ e6fd68fa-e8d8-11ea-3dc4-274caceda222 +md"# So, what *is* an image?" + +# ╔═╡ 63a1d282-e8d5-11ea-0bba-b9cdd32a218b +typeof(philip) + +# ╔═╡ fc5e1af0-e8d8-11ea-1077-07216ff96d29 +md""" +- It's an `Array` + +- The `2` means that it has **2 dimensions** (a **matrix**) + +$(HTML("
")) + +- `RGBX{Normed{UInt8,8}}` is the type of object stored in the array + +- A Julia object representing a colour + +- RGB = Red, Green, Blue +""" + +# ╔═╡ c79dd836-e8e8-11ea-029d-57be9899979a +md"## Getting pieces of an image" + + + +# ╔═╡ ae260168-e932-11ea-38fd-4f2c6f43e21c +begin + (h, w) = size(philip) + head = philip[(h ÷ 2):h, (w ÷ 10): (9w ÷ 10)] + # `÷` is typed as \div -- integer division +end + +# ╔═╡ 47d1bc04-ebcb-11ea-3643-d1ba8dea57c8 +size(head) + +# ╔═╡ 72400458-ebcb-11ea-26b6-678ae1de8e23 +size(philip) + +# ╔═╡ f57ea7c2-e932-11ea-0d52-4112187bcb38 +md"## Manipulating matrices + +- An image is just a matrix, so we can manipulate *matrices* to manipulate the *image* +" + +# ╔═╡ 740ed2e2-e933-11ea-236c-f3c3f09d0f8b +[head head] + +# ╔═╡ 6128a5ba-e93b-11ea-03f5-f170c7b90b25 +md"##" + +# ╔═╡ 78eafe4e-e933-11ea-3539-c13feb894ef6 +[ + head reverse(head, dims=2) + reverse(head, dims=1) reverse(reverse(head, dims=1), dims=2) +] + +# ╔═╡ bf3f9050-e933-11ea-0df7-e5dcff6bb3ee +md"## Manipulating an image + +- How can we get inside the image and change it? + +- There are two possibilities: + + - **Modify** (**mutate**) numbers inside the array -- useful to change a small piece + + - Create a new **copy** of the array -- useful to alter everything together +" + +# ╔═╡ 212e1f12-e934-11ea-2f35-51c7a6c8dff1 +md"## Painting a piece of an image + +- Let's paint a corner red + +- We'll copy the image first so we don't destroy the original +" + +# ╔═╡ 117a98c0-e936-11ea-3aac-8f66337cea68 +new_phil = copy(head) + +# ╔═╡ 8004d076-e93b-11ea-29cc-a1bfcc75e87f +md"##" + +# ╔═╡ 3ac63296-e936-11ea-2144-f94bdbd60eaf +red = RGB(1, 0, 0) + +# ╔═╡ 3e3f841a-e936-11ea-0a81-1b95fe0faa83 +for i in 1:100 + for j in 1:300 + new_phil[i, j] = red + end +end + +# ╔═╡ 5978db50-e936-11ea-3145-059a51be2281 +md"Note that `for` loops *do not return anything* (or, rather, they return `nothing`)" + +# ╔═╡ 21638b14-ebcc-11ea-1761-bbd2f4306a96 +new_phil + +# ╔═╡ 70cb0e36-e936-11ea-3ade-49fde77cb696 +md"""## Element-wise operations: "Broadcasting" + +- Julia provides powerful technology for operating element by element: **broadcasting** + +- Adding "`.`" applies an operation element by element +""" + +# ╔═╡ b3ea975e-e936-11ea-067d-81339575a3cb +begin + new_phil2 = copy(new_phil) + new_phil2[100:200, 1:100] .= RGB(0, 1, 0) + new_phil2 +end + +# ╔═╡ 918a0762-e93b-11ea-1115-71dbfdb03f27 +md"##" + +# ╔═╡ daabe66c-e937-11ea-3bc3-d77f2bce406c +new_phil2 + +# ╔═╡ 095ced62-e938-11ea-1169-939dc7136fd0 +md"## Modifying the whole image at once + +- We can use the same trick to modify the whole image at once + +- Let's **redify** the image + +- We define a **function** that turns a colour into just its red component +" + +# ╔═╡ 31f3605a-e938-11ea-3a6d-29a185bbee31 +function redify(c) + return RGB(c.r, 0, 0) +end + +# ╔═╡ 2744a556-e94f-11ea-2434-d53c24e59285 +begin + color = RGB(0.9, 0.7, 0.2) + + [color, redify(color)] +end + +# ╔═╡ 98412a36-e93b-11ea-1954-f1c105c6ed4a +md"##" + +# ╔═╡ 3c32efde-e938-11ea-1ae4-5d88290f5311 +redify.(philip) + +# ╔═╡ 4b26e4e6-e938-11ea-2635-6d4fc15e13b7 +md"## Transforming an image + +- The main goal of this week will be to transfrom images in more interesting ways + +- First let's **decimate** poor Phil +" + + + +# ╔═╡ c12e0928-e93b-11ea-0922-2b590a99ee89 +md"##" + +# ╔═╡ ff5dc538-e938-11ea-058f-693d6b016640 +md"## Experiments come alive with interaction + +- We start to get a feel for things when we can **experiment**! +" + +# ╔═╡ fa24f4a8-e93b-11ea-06bd-25c9672166d6 +md"##" + +# ╔═╡ 15ce202e-e939-11ea-2387-93be0ec4cf1f +@bind repeat_count Slider(1:10, show_value=true) + +# ╔═╡ bf2167a4-e93d-11ea-03b2-cdd24b459ba9 +md"## Summary + +- Images are readily-accessible data about the world + +- We want to process them to extract information + +- Relatively simple mathematical operations can transform images in useful ways +" + +# ╔═╡ 58184d88-e939-11ea-2fc8-73b3476ebe92 +expand(image, ratio=5) = kron(image, ones(ratio, ratio)) + +# ╔═╡ 2dd09f16-e93a-11ea-2cdc-13f558e3391d +extract_red(c) = c.r + +# ╔═╡ df1b7996-e93b-11ea-1a3a-81b4ec520679 +decimate(image, ratio=5) = image[1:ratio:end, 1:ratio:end] + +# ╔═╡ 41fa85c0-e939-11ea-1ad8-79805a2083bb +poor_phil = decimate(head, 5) + +# ╔═╡ cd5721d0-ede6-11ea-0918-1992c69bccc6 +repeat(poor_phil, repeat_count, repeat_count) + +# ╔═╡ b8daeea0-ec79-11ea-34b5-3f13e8a56a42 +md"# Appendix" + +# ╔═╡ bf1bb2c8-ec79-11ea-0671-3ffb34828f3c +md"## Package environment" + +# ╔═╡ 69e3aa82-e93c-11ea-23fe-c1103d989cba +md"## Camera input" + +# ╔═╡ 739c3bb6-e93c-11ea-127b-efb6a8ab9379 +function camera_input(;max_size=200, default_url="https://i.imgur.com/SUmi94P.png") +""" + + + +
+
+ + +
+ +
+ +
+
+ +
+ + Enable webcam + +
+ + + +""" |> HTML +end + + +# ╔═╡ 9529bc40-e93c-11ea-2587-3186e0978476 +@bind raw_camera_data camera_input(;max_size=2000) + +# ╔═╡ 832ebd1a-e93c-11ea-1d18-d784f3184ebe + +function process_raw_camera_data(raw_camera_data) + # the raw image data is a long byte array, we need to transform it into something + # more "Julian" - something with more _structure_. + + # The encoding of the raw byte stream is: + # every 4 bytes is a single pixel + # every pixel has 4 values: Red, Green, Blue, Alpha + # (we ignore alpha for this notebook) + + # So to get the red values for each pixel, we take every 4th value, starting at + # the 1st: + reds_flat = UInt8.(raw_camera_data["data"][1:4:end]) + greens_flat = UInt8.(raw_camera_data["data"][2:4:end]) + blues_flat = UInt8.(raw_camera_data["data"][3:4:end]) + + # but these are still 1-dimensional arrays, nicknamed 'flat' arrays + # We will 'reshape' this into 2D arrays: + + width = raw_camera_data["width"] + height = raw_camera_data["height"] + + # shuffle and flip to get it in the right shape + reds = reshape(reds_flat, (width, height))' / 255.0 + greens = reshape(greens_flat, (width, height))' / 255.0 + blues = reshape(blues_flat, (width, height))' / 255.0 + + # we have our 2D array for each color + # Let's create a single 2D array, where each value contains the R, G and B value of + # that pixel + + RGB.(reds, greens, blues) +end + +# ╔═╡ 9a843af8-e93c-11ea-311b-1bc6d5b58492 +grant = decimate(process_raw_camera_data(raw_camera_data), 2) + +# ╔═╡ 6aa73286-ede7-11ea-232b-63e052222ecd +[ + grant grant[:,end:-1:1] + grant[end:-1:1,:] grant[end:-1:1,end:-1:1] +] + +# ╔═╡ Cell order: +# ╟─a50b5f48-e8d5-11ea-1f05-a3741b5d15ba +# ╟─8a6fed4c-e94b-11ea-1113-d56f56fb293b +# ╟─dc53f316-e8c8-11ea-150f-1374dbce114a +# ╟─c3f43d66-e94b-11ea-02bd-23cfeb878ff1 +# ╟─c6c77738-e94b-11ea-22f5-1dce3dbcc3ca +# ╟─cf80793a-e94b-11ea-0120-f7913ae06f22 +# ╟─d1638d96-e94b-11ea-2ff4-910e399f864d +# ╟─0117246a-e94c-11ea-1a76-c981ce8e725d +# ╟─27060098-e8c9-11ea-2fe0-03b39b1ddc32 +# ╟─4fc58814-e94b-11ea-339b-cb714a63f9b6 +# ╟─f067d3b8-e8c8-11ea-20cb-474709ffa99a +# ╠═37c1d012-ebc9-11ea-2dfe-8b86bb78f283 +# ╟─a0a97214-e8d2-11ea-0f46-0bfaf016ab6d +# ╟─1697a756-e93d-11ea-0b6e-c9c78d527993 +# ╟─af28faca-ebb7-11ea-130d-0f94bf9bd836 +# ╠═9529bc40-e93c-11ea-2587-3186e0978476 +# ╟─ee1d1596-e94a-11ea-0fb4-cd05f62471d3 +# ╠═6aa73286-ede7-11ea-232b-63e052222ecd +# ╠═9a843af8-e93c-11ea-311b-1bc6d5b58492 +# ╟─8ab9a978-e8c9-11ea-2476-f1ef4ba1b619 +# ╟─38c54bfc-e8cb-11ea-3d52-0f02452f8ba1 +# ╟─983f8270-e8c9-11ea-29d2-adeccb5a7ffc +# ╟─2fcaef88-e8ca-11ea-23f7-29c48580f43c +# ╟─7636c4b0-e8d1-11ea-2051-757a850a9d30 +# ╟─bca22176-e8ca-11ea-2004-ebeb103116b5 +# ╟─0ad91f1e-e8d2-11ea-2c18-93f66c906a8b +# ╠═de373816-ec79-11ea-2772-ebdca52246ac +# ╠═552129ae-ebca-11ea-1fa1-3f9fa00a2601 +# ╠═54c1ba3c-e8d2-11ea-3564-bdaca8563738 +# ╠═6e0fefb6-e8d4-11ea-1f9b-e7a3db40df39 +# ╠═9c359212-ec79-11ea-2d7e-0124dad5f127 +# ╠═7703b032-ebca-11ea-3074-0b80a077078e +# ╠═7eff3522-ebca-11ea-1a65-59e66a4e72ab +# ╠═c9cd6c04-ebca-11ea-0990-5fa19ff7ed97 +# ╟─0d873d9c-e93b-11ea-2425-1bd79677fb97 +# ╠═6b09354a-ebb9-11ea-2d5a-3b75c5ae7aa9 +# ╟─2d6c434e-e93b-11ea-2678-3b9db4975089 +# ╠═2b14e93e-e93b-11ea-25f1-5f565f80e778 +# ╟─0bdc6058-e8d5-11ea-1889-3f706cea7a1f +# ╠═e61db924-ebca-11ea-2f79-f9f1c121b7f5 +# ╠═ef60fcc4-ebca-11ea-3f69-155afffe8ea8 +# ╠═fac550ec-ebca-11ea-337a-dbc16848c617 +# ╟─42aa8cfe-e8d5-11ea-3cb9-c365b98e7a8c +# ╠═4eea5710-e8d5-11ea-3978-af66ee2a137e +# ╟─57b3a0c2-e8d5-11ea-15aa-8da4549f849b +# ╠═03a7c0fc-ebba-11ea-1c71-79d750c97b16 +# ╟─e6fd68fa-e8d8-11ea-3dc4-274caceda222 +# ╠═63a1d282-e8d5-11ea-0bba-b9cdd32a218b +# ╟─fc5e1af0-e8d8-11ea-1077-07216ff96d29 +# ╟─c79dd836-e8e8-11ea-029d-57be9899979a +# ╠═ae260168-e932-11ea-38fd-4f2c6f43e21c +# ╠═47d1bc04-ebcb-11ea-3643-d1ba8dea57c8 +# ╠═72400458-ebcb-11ea-26b6-678ae1de8e23 +# ╟─f57ea7c2-e932-11ea-0d52-4112187bcb38 +# ╠═740ed2e2-e933-11ea-236c-f3c3f09d0f8b +# ╟─6128a5ba-e93b-11ea-03f5-f170c7b90b25 +# ╠═78eafe4e-e933-11ea-3539-c13feb894ef6 +# ╟─bf3f9050-e933-11ea-0df7-e5dcff6bb3ee +# ╟─212e1f12-e934-11ea-2f35-51c7a6c8dff1 +# ╠═117a98c0-e936-11ea-3aac-8f66337cea68 +# ╟─8004d076-e93b-11ea-29cc-a1bfcc75e87f +# ╠═3ac63296-e936-11ea-2144-f94bdbd60eaf +# ╠═3e3f841a-e936-11ea-0a81-1b95fe0faa83 +# ╟─5978db50-e936-11ea-3145-059a51be2281 +# ╠═21638b14-ebcc-11ea-1761-bbd2f4306a96 +# ╟─70cb0e36-e936-11ea-3ade-49fde77cb696 +# ╠═b3ea975e-e936-11ea-067d-81339575a3cb +# ╟─918a0762-e93b-11ea-1115-71dbfdb03f27 +# ╠═daabe66c-e937-11ea-3bc3-d77f2bce406c +# ╟─095ced62-e938-11ea-1169-939dc7136fd0 +# ╠═31f3605a-e938-11ea-3a6d-29a185bbee31 +# ╠═2744a556-e94f-11ea-2434-d53c24e59285 +# ╟─98412a36-e93b-11ea-1954-f1c105c6ed4a +# ╠═3c32efde-e938-11ea-1ae4-5d88290f5311 +# ╟─4b26e4e6-e938-11ea-2635-6d4fc15e13b7 +# ╠═41fa85c0-e939-11ea-1ad8-79805a2083bb +# ╟─c12e0928-e93b-11ea-0922-2b590a99ee89 +# ╟─ff5dc538-e938-11ea-058f-693d6b016640 +# ╠═fbe11200-e938-11ea-12e9-6125c1b56b25 +# ╟─fa24f4a8-e93b-11ea-06bd-25c9672166d6 +# ╠═15ce202e-e939-11ea-2387-93be0ec4cf1f +# ╠═cd5721d0-ede6-11ea-0918-1992c69bccc6 +# ╟─bf2167a4-e93d-11ea-03b2-cdd24b459ba9 +# ╟─5e688928-e939-11ea-0e16-fbc80af390ab +# ╟─58184d88-e939-11ea-2fc8-73b3476ebe92 +# ╟─2dd09f16-e93a-11ea-2cdc-13f558e3391d +# ╟─df1b7996-e93b-11ea-1a3a-81b4ec520679 +# ╟─b8daeea0-ec79-11ea-34b5-3f13e8a56a42 +# ╟─bf1bb2c8-ec79-11ea-0671-3ffb34828f3c +# ╟─69e3aa82-e93c-11ea-23fe-c1103d989cba +# ╟─739c3bb6-e93c-11ea-127b-efb6a8ab9379 +# ╟─832ebd1a-e93c-11ea-1d18-d784f3184ebe diff --git a/Wonderful invention.jl b/Wonderful invention.jl new file mode 100644 index 0000000..8ff1f95 --- /dev/null +++ b/Wonderful invention.jl @@ -0,0 +1,256 @@ +### A Pluto.jl notebook ### +# v0.12.11 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ 15a4ba3e-f0d1-11ea-2ef1-5ff1dee8795f +using Pkg + +# ╔═╡ 21e744b8-f0d1-11ea-2e09-7ffbcdf43c37 +begin + Pkg.add("Gadfly") + Pkg.add("Compose") + Pkg.add("Statistics") + Pkg.add("Hyperscript") + Pkg.add("Colors") + Pkg.add("Images") + Pkg.add("ImageMagick") + Pkg.add("ImageFiltering") + + using Gadfly + using Images + using Compose + using Hyperscript + using Colors + using Statistics + using PlutoUI + using ImageMagick + using ImageFiltering +end + +# ╔═╡ 1ab1c808-f0d1-11ea-03a7-e9854427d45f +Pkg.activate(mktempdir()) + +# ╔═╡ 10f850fc-f0d1-11ea-2a58-2326a9ea1e2a +set_default_plot_size(12cm, 12cm) + +# ╔═╡ 7b4d5270-f0d3-11ea-0b48-79005f20602c +function convolve(M, kernel) + height, width = size(kernel) + + half_height = height ÷ 2 + half_width = width ÷ 2 + + new_image = similar(M) + + # (i, j) loop over the original image + m, n = size(M) + @inbounds for i in 1:m + for j in 1:n + # (k, l) loop over the neighbouring pixels + accumulator = 0 * M[1, 1] + for k in -half_height:-half_height + height - 1 + for l in -half_width:-half_width + width - 1 + Mi = i - k + Mj = j - l + # First index into M + if Mi < 1 + Mi = 1 + elseif Mi > m + Mi = m + end + # Second index into M + if Mj < 1 + Mj = 1 + elseif Mj > n + Mj = n + end + + accumulator += kernel[k, l] * M[Mi, Mj] + end + end + new_image[i, j] = accumulator + end + end + + return new_image +end + +# ╔═╡ 6fd3b7a4-f0d3-11ea-1f26-fb9740cd16e0 +function disc(n, r1=0.8, r2=0.8) + white = RGB{Float64}(1,1,1) + blue = RGB{Float64}(colorant"#4EC0E3") + convolve( + [(i-n/2)^2 + (j-n/2)^2 <= (n/2-5)^2 ? white : blue for i=1:n, j=1:n], + Kernel.gaussian((1,1)) + ) +end + +# ╔═╡ fe3559e0-f13b-11ea-06c8-a314e44c20d6 +brightness(c) = 0.3 * c.r + 0.59 * c.g + 0.11 * c.b + +# ╔═╡ 0ccf76e4-f0d9-11ea-07c9-0159e3d4d733 +@bind img_select Radio(["disc", "mario"], default="disc") + +# ╔═╡ 236dab08-f13d-11ea-1922-a3b82cfc7f51 +begin + url = "http://files.softicons.com/download/game-icons/super-mario-icons-by-sandro-pereira/png/32/Retro%20Mario.png" + img = Dict( + "disc" => disc(25), + "mario" => load(download(url)) + )[img_select] +end + +# ╔═╡ 03434682-f13b-11ea-2b6e-11ad781e9a51 +md"""Show $G_x$ $(@bind Gx CheckBox()) + + Show $G_y$ $(@bind Gy CheckBox())""" + +# ╔═╡ ca13597a-f168-11ea-1a2c-ff7b98b7b2c7 +function partial_derivatives(img) + Sy,Sx = Kernel.sobel() + ∇x, ∇y = zeros(size(img)), zeros(size(img)) + + if Gx + ∇x = convolve(brightness.(img), Sx) + end + if Gy + ∇y = convolve(brightness.(img), Sy) + end + return ∇x, ∇y +end + +# ╔═╡ b369584c-f183-11ea-260a-35dc797e63ad + + +# ╔═╡ b2cbe058-f183-11ea-39dc-23d4a5b92796 + + +# ╔═╡ 9d9cccb2-f118-11ea-1638-c76682e636b2 +function arrowhead(θ) + eq_triangle = [(0, 1/sqrt(3)), + (-1/3, -2/(2 * sqrt(3))), + (1/3, -2/(2 * sqrt(3)))] + + compose(context(units=UnitBox(-1,-1,2,2), rotation=Rotation(θ, 0, 0)), + polygon(eq_triangle)) +end + +# ╔═╡ b7ea8a28-f0d7-11ea-3e98-7b19a1f58304 +function quiver(points, vecs) + xmin = minimum(first.(points)) + ymin = minimum(last.(points)) + xmax = maximum(first.(points)) + ymax = maximum(last.(points)) + hs = map(x->hypot(x...), vecs) + hs = hs / maximum(hs) + + vector(p, v, h) = all(iszero, v) ? context() : + (context(), + (context((p.+v.*6 .- .2)..., .4,.4), + arrowhead(atan(v[2], v[1]) - pi/2)), + stroke(RGBA(90/255,39/255,41/255,h)), + fill(RGBA(90/255,39/255,41/255,h)), + line([p, p.+v.*8])) + + compose(context(units=UnitBox(xmin,ymin,xmax,ymax)), + vector.(points, vecs, hs)...) +end + +# ╔═╡ c821b906-f0d8-11ea-2df0-8f2d06964aa2 +function sobel_quiver(img, ∇x, ∇y) + quiver([(j-1,i-1) for i=1:size(img,1), j=1:size(img,2)], + [(∇x[i,j], ∇y[i,j]) for i=1:size(img,1), j=1:size(img,2)]) +end + +# ╔═╡ 6da3fdfe-f0dd-11ea-2407-7b85217b35cc +# render an Image using squares in Compose +function compimg(img) + xmax, ymax = size(img) + xmin, ymin = 0, 0 + arr = [(j-1, i-1) for i=1:ymax, j=1:xmax] + + compose(context(units=UnitBox(xmin, ymin, xmax, ymax)), + fill(vec(img)), + rectangle( + first.(arr), + last.(arr), + fill(1.0, length(arr)), + fill(1.0, length(arr)))) +end + +# ╔═╡ f22aa34e-f0df-11ea-3053-3dcdc070ec2f +let + ∇x, ∇y = partial_derivatives(img) + + compose(context(), + sobel_quiver(img, ∇x, ∇y), + compimg(img)) +end + +# ╔═╡ 885ec336-f146-11ea-00c4-c1d1ab4c0001 + function show_colored_array(array) + pos_color = RGB(0.36, 0.82, 0.8) + neg_color = RGB(0.99, 0.18, 0.13) + to_rgb(x) = max(x, 0) * pos_color + max(-x, 0) * neg_color + to_rgb.(array) / maximum(abs.(array)) + end + +# ╔═╡ 9232dcc8-f188-11ea-08fe-b787ea93c598 +begin + Sy, Sx = Kernel.sobel() + show_colored_array(Sx) + Sx +end + +# ╔═╡ 7864bd00-f146-11ea-0020-7fccb3913d8b +let + ∇x, ∇y = partial_derivatives(img) + + to_show = (x -> RGB(0, 0, 0)).(zeros(size(img))) + if Gx && Gy + edged = sqrt.(∇x.^2 + ∇y.^2) + to_show = Gray.(edged) / maximum(edged) + elseif Gx + to_show = show_colored_array(∇x) + elseif Gy + to_show = show_colored_array(∇y) + end + compose( + context(), + compimg(to_show) + ) +end + +# ╔═╡ Cell order: +# ╠═15a4ba3e-f0d1-11ea-2ef1-5ff1dee8795f +# ╠═1ab1c808-f0d1-11ea-03a7-e9854427d45f +# ╠═21e744b8-f0d1-11ea-2e09-7ffbcdf43c37 +# ╠═10f850fc-f0d1-11ea-2a58-2326a9ea1e2a +# ╟─7b4d5270-f0d3-11ea-0b48-79005f20602c +# ╠═6fd3b7a4-f0d3-11ea-1f26-fb9740cd16e0 +# ╠═fe3559e0-f13b-11ea-06c8-a314e44c20d6 +# ╟─b7ea8a28-f0d7-11ea-3e98-7b19a1f58304 +# ╟─0ccf76e4-f0d9-11ea-07c9-0159e3d4d733 +# ╟─236dab08-f13d-11ea-1922-a3b82cfc7f51 +# ╟─03434682-f13b-11ea-2b6e-11ad781e9a51 +# ╠═ca13597a-f168-11ea-1a2c-ff7b98b7b2c7 +# ╟─f22aa34e-f0df-11ea-3053-3dcdc070ec2f +# ╟─9232dcc8-f188-11ea-08fe-b787ea93c598 +# ╠═7864bd00-f146-11ea-0020-7fccb3913d8b +# ╠═b369584c-f183-11ea-260a-35dc797e63ad +# ╠═b2cbe058-f183-11ea-39dc-23d4a5b92796 +# ╟─9d9cccb2-f118-11ea-1638-c76682e636b2 +# ╟─c821b906-f0d8-11ea-2df0-8f2d06964aa2 +# ╟─6da3fdfe-f0dd-11ea-2407-7b85217b35cc +# ╠═885ec336-f146-11ea-00c4-c1d1ab4c0001 diff --git a/gradient.jl b/gradient.jl new file mode 100644 index 0000000..fb361fc --- /dev/null +++ b/gradient.jl @@ -0,0 +1,256 @@ +### A Pluto.jl notebook ### +# v0.11.10 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ 15a4ba3e-f0d1-11ea-2ef1-5ff1dee8795f +using Pkg + +# ╔═╡ 21e744b8-f0d1-11ea-2e09-7ffbcdf43c37 +begin + Pkg.add("Gadfly") + Pkg.add("Compose") + Pkg.add("Statistics") + Pkg.add("Hyperscript") + Pkg.add("Colors") + Pkg.add("Images") + Pkg.add("ImageMagick") + Pkg.add("ImageFiltering") + + using Gadfly + using Images + using Compose + using Hyperscript + using Colors + using Statistics + using PlutoUI + using ImageMagick + using ImageFiltering +end + +# ╔═╡ 1ab1c808-f0d1-11ea-03a7-e9854427d45f +Pkg.activate(mktempdir()) + +# ╔═╡ 10f850fc-f0d1-11ea-2a58-2326a9ea1e2a +set_default_plot_size(12cm, 12cm) + +# ╔═╡ 7b4d5270-f0d3-11ea-0b48-79005f20602c +function convolve(M, kernel) + height, width = size(kernel) + + half_height = height ÷ 2 + half_width = width ÷ 2 + + new_image = similar(M) + + # (i, j) loop over the original image + m, n = size(M) + @inbounds for i in 1:m + for j in 1:n + # (k, l) loop over the neighbouring pixels + accumulator = 0 * M[1, 1] + for k in -half_height:-half_height + height - 1 + for l in -half_width:-half_width + width - 1 + Mi = i - k + Mj = j - l + # First index into M + if Mi < 1 + Mi = 1 + elseif Mi > m + Mi = m + end + # Second index into M + if Mj < 1 + Mj = 1 + elseif Mj > n + Mj = n + end + + accumulator += kernel[k, l] * M[Mi, Mj] + end + end + new_image[i, j] = accumulator + end + end + + return new_image +end + +# ╔═╡ 6fd3b7a4-f0d3-11ea-1f26-fb9740cd16e0 +function disc(n, r1=0.8, r2=0.8) + white = RGB{Float64}(1,1,1) + blue = RGB{Float64}(colorant"#4EC0E3") + convolve( + [(i-n/2)^2 + (j-n/2)^2 <= (n/2-5)^2 ? white : blue for i=1:n, j=1:n], + Kernel.gaussian((1,1)) + ) +end + +# ╔═╡ fe3559e0-f13b-11ea-06c8-a314e44c20d6 +brightness(c) = 0.3 * c.r + 0.59 * c.g + 0.11 * c.b + +# ╔═╡ 0ccf76e4-f0d9-11ea-07c9-0159e3d4d733 +@bind img_select Radio(["disc", "mario"], default="disc") + +# ╔═╡ 236dab08-f13d-11ea-1922-a3b82cfc7f51 +begin + url = "http://files.softicons.com/download/game-icons/super-mario-icons-by-sandro-pereira/png/32/Retro%20Mario.png" + img = Dict( + "disc" => disc(25), + "mario" => load(download(url)) + )[img_select] +end + +# ╔═╡ 03434682-f13b-11ea-2b6e-11ad781e9a51 +md"""Show $G_x$ $(@bind Gx CheckBox()) + + Show $G_y$ $(@bind Gy CheckBox())""" + +# ╔═╡ ca13597a-f168-11ea-1a2c-ff7b98b7b2c7 +function partial_derivatives(img) + Sy,Sx = Kernel.sobel() + ∇x, ∇y = zeros(size(img)), zeros(size(img)) + + if Gx + ∇x = convolve(brightness.(img), Sx) + end + if Gy + ∇y = convolve(brightness.(img), Sy) + end + return ∇x, ∇y +end + +# ╔═╡ b369584c-f183-11ea-260a-35dc797e63ad + + +# ╔═╡ b2cbe058-f183-11ea-39dc-23d4a5b92796 + + +# ╔═╡ 9d9cccb2-f118-11ea-1638-c76682e636b2 +function arrowhead(θ) + eq_triangle = [(0, 1/sqrt(3)), + (-1/3, -2/(2 * sqrt(3))), + (1/3, -2/(2 * sqrt(3)))] + + compose(context(units=UnitBox(-1,-1,2,2), rotation=Rotation(θ, 0, 0)), + polygon(eq_triangle)) +end + +# ╔═╡ b7ea8a28-f0d7-11ea-3e98-7b19a1f58304 +function quiver(points, vecs) + xmin = minimum(first.(points)) + ymin = minimum(last.(points)) + xmax = maximum(first.(points)) + ymax = maximum(last.(points)) + hs = map(x->hypot(x...), vecs) + hs = hs / maximum(hs) + + vector(p, v, h) = all(iszero, v) ? context() : + (context(), + (context((p.+v.*6 .- .2)..., .4,.4), + arrowhead(atan(v[2], v[1]) - pi/2)), + stroke(RGBA(90/255,39/255,41/255,h)), + fill(RGBA(90/255,39/255,41/255,h)), + line([p, p.+v.*8])) + + compose(context(units=UnitBox(xmin,ymin,xmax,ymax)), + vector.(points, vecs, hs)...) +end + +# ╔═╡ c821b906-f0d8-11ea-2df0-8f2d06964aa2 +function sobel_quiver(img, ∇x, ∇y) + quiver([(j-1,i-1) for i=1:size(img,1), j=1:size(img,2)], + [(∇x[i,j], ∇y[i,j]) for i=1:size(img,1), j=1:size(img,2)]) +end + +# ╔═╡ 6da3fdfe-f0dd-11ea-2407-7b85217b35cc +# render an Image using squares in Compose +function compimg(img) + xmax, ymax = size(img) + xmin, ymin = 0, 0 + arr = [(j-1, i-1) for i=1:ymax, j=1:xmax] + + compose(context(units=UnitBox(xmin, ymin, xmax, ymax)), + fill(vec(img)), + rectangle( + first.(arr), + last.(arr), + fill(1.0, length(arr)), + fill(1.0, length(arr)))) +end + +# ╔═╡ f22aa34e-f0df-11ea-3053-3dcdc070ec2f +let + ∇x, ∇y = partial_derivatives(img) + + compose(context(), + sobel_quiver(img, ∇x, ∇y), + compimg(img)) +end + +# ╔═╡ 885ec336-f146-11ea-00c4-c1d1ab4c0001 + function show_colored_array(array) + pos_color = RGB(0.36, 0.82, 0.8) + neg_color = RGB(0.99, 0.18, 0.13) + to_rgb(x) = max(x, 0) * pos_color + max(-x, 0) * neg_color + to_rgb.(array) / maximum(abs.(array)) + end + +# ╔═╡ 9232dcc8-f188-11ea-08fe-b787ea93c598 +begin + Sy, Sx = Kernel.sobel() + show_colored_array(Sx) + Sx +end + +# ╔═╡ 7864bd00-f146-11ea-0020-7fccb3913d8b +let + ∇x, ∇y = partial_derivatives(img) + + to_show = (x -> RGB(0, 0, 0)).(zeros(size(img))) + if Gx && Gy + edged = sqrt.(∇x.^2 + ∇y.^2) + to_show = Gray.(edged) / maximum(edged) + elseif Gx + to_show = show_colored_array(∇x) + elseif Gy + to_show = show_colored_array(∇y) + end + compose( + context(), + compimg(to_show) + ) +end + +# ╔═╡ Cell order: +# ╠═15a4ba3e-f0d1-11ea-2ef1-5ff1dee8795f +# ╠═1ab1c808-f0d1-11ea-03a7-e9854427d45f +# ╟─21e744b8-f0d1-11ea-2e09-7ffbcdf43c37 +# ╠═10f850fc-f0d1-11ea-2a58-2326a9ea1e2a +# ╟─7b4d5270-f0d3-11ea-0b48-79005f20602c +# ╠═6fd3b7a4-f0d3-11ea-1f26-fb9740cd16e0 +# ╟─fe3559e0-f13b-11ea-06c8-a314e44c20d6 +# ╟─b7ea8a28-f0d7-11ea-3e98-7b19a1f58304 +# ╟─0ccf76e4-f0d9-11ea-07c9-0159e3d4d733 +# ╟─236dab08-f13d-11ea-1922-a3b82cfc7f51 +# ╟─03434682-f13b-11ea-2b6e-11ad781e9a51 +# ╟─ca13597a-f168-11ea-1a2c-ff7b98b7b2c7 +# ╟─f22aa34e-f0df-11ea-3053-3dcdc070ec2f +# ╟─9232dcc8-f188-11ea-08fe-b787ea93c598 +# ╠═7864bd00-f146-11ea-0020-7fccb3913d8b +# ╠═b369584c-f183-11ea-260a-35dc797e63ad +# ╠═b2cbe058-f183-11ea-39dc-23d4a5b92796 +# ╟─9d9cccb2-f118-11ea-1638-c76682e636b2 +# ╟─c821b906-f0d8-11ea-2df0-8f2d06964aa2 +# ╟─6da3fdfe-f0dd-11ea-2407-7b85217b35cc +# ╠═885ec336-f146-11ea-00c4-c1d1ab4c0001 diff --git a/hw0.jl b/hw0.jl new file mode 100644 index 0000000..fc7f5ca --- /dev/null +++ b/hw0.jl @@ -0,0 +1,359 @@ +### A Pluto.jl notebook ### +# v0.12.11 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ fafae38e-e852-11ea-1208-732b4744e4c2 +md"_homework 0, version 2_" + +# ╔═╡ 7308bc54-e6cd-11ea-0eab-83f7535edf25 +# edit the code below to set your name and kerberos ID (i.e. email without @mit.edu) + +student = (name = "Jazzy Doe", kerberos_id = "jazz") + +# press the ▶ button in the bottom right of this cell to run your edits +# or use Shift+Enter + +# you might need to wait until all other cells in this notebook have completed running. +# scroll down the page to see what's up + +# ╔═╡ cdff6730-e785-11ea-2546-4969521b33a7 +md""" + +Submission by: **_$(student.name)_** ($(student.kerberos_id)@mit.edu) +""" + +# ╔═╡ a2181260-e6cd-11ea-2a69-8d9d31d1ef0e +md""" +# Homework 0: Getting up and running + +First of all, **_welcome to the course!_** We are excited to teach you about real world applications of scientific computing, using the same tools that we work with ourselves. + +Before we start next week, we'd like everyone to **submit this zeroth homework assignment**. It will not affect your grade, but it will help us get everything running smoothly when the course starts. If you're stuck or don't have much time, just fill in your name and ID and submit 🙂 +""" + +# ╔═╡ 094e39c8-e6ce-11ea-131b-07c4a1199edf + + +# ╔═╡ 31a8fbf8-e6ce-11ea-2c66-4b4d02b41995 + + +# ╔═╡ 339c2d5c-e6ce-11ea-32f9-714b3628909c +md"## Exercise 1 - _Square root by Newton's method_ + +Computing the square of a number is easy -- you just multiply it with itself. + +But how does one compute the square root of a number? + +##### Algorithm: + +Given: $x$ + +Output: $\sqrt{x}$ + +1. Take a guess `a` +1. Divide `x` by `a` +1. Set a = the average of `x/a` and `a`. (The square root must be between these two numbers. Why?) +1. Repeat until `x/a` is roughly equal to `a`. Return `a` as the square root. + +In general, you will never get to the point where `x/a` is _exactly_ equal to `a`. So if our algorithm keeps going until `x/a == a`, then it will get stuck. + +So instead, the algorithm takes a parameter `error_margin`, which is used to decide when `x/a` and `a` are close enough to halt. +" + +# ╔═╡ 56866718-e6ce-11ea-0804-d108af4e5653 +md"### Exercise 1.1 + +Step 3 in the algorithm sets the new guess to be the average of `x/a` and the old guess `a`. + +This is because the square root must be between the numbers `x/a` and `a`. Why? +" + +# ╔═╡ bccf0e88-e754-11ea-3ab8-0170c2d44628 +ex_1_1 = md""" +your answer here +""" + +# you might need to wait until all other cells in this notebook have completed running. +# scroll down the page to see what's up + +# ╔═╡ e7abd366-e7a6-11ea-30d7-1b6194614d0a +if !(@isdefined ex_1_1) + md"""Do not change the name of the variable - write you answer as `ex_1_1 = "..."`""" +end + +# ╔═╡ d62f223c-e754-11ea-2470-e72a605a9d7e +md"### Exercise 1.2 + +Write a function newton_sqrt(x) which implements the above algorithm." + +# ╔═╡ 4896bf0c-e754-11ea-19dc-1380bb356ab6 +function newton_sqrt(x, error_margin=0.01, a=x / 2) # a=x/2 is the default value of `a` + while true + xa = x / a + if (abs(a - xa) < error_margin) + return a + end + a = (xa + a) / 2 + end +end + +# ╔═╡ 7a01a508-e78a-11ea-11da-999d38785348 +newton_sqrt(2) + +# ╔═╡ 682db9f8-e7b1-11ea-3949-6b683ca8b47b +let + result = newton_sqrt(2, 0.01) + if !(result isa Number) + md""" +!!! warning "Not a number" + `newton_sqrt` did not return a number. Did you forget to write `return`? + """ + elseif abs(result - sqrt(2)) < 0.01 + md""" +!!! correct + Well done! + """ + else + md""" +!!! warning "Incorrect" + Keep working on it! + """ + end +end + +# ╔═╡ 088cc652-e7a8-11ea-0ca7-f744f6f3afdd +md""" +!!! hint + `abs(r - s)` is the distance between `r` and `s` +""" + +# ╔═╡ c18dce7a-e7a7-11ea-0a1a-f944d46754e5 +md""" +!!! hint + If you're stuck, feel free to cheat, this is homework 0 after all 🙃 + + Julia has a function called `sqrt` +""" + +# ╔═╡ 5e24d95c-e6ce-11ea-24be-bb19e1e14657 +md"## Exercise 2 - _Sierpinksi's triangle_ + +Sierpinski's triangle is defined _recursively_: + +- Sierpinski's triangle of complexity N is a figure in the form of a triangle which is made of 3 triangular figures which are themselves Sierpinski's triangles of complexity N-1. + +- A Sierpinski's triangle of complexity 0 is a simple solid equilateral triangle +" + +# ╔═╡ 6b8883f6-e7b3-11ea-155e-6f62117e123b +md"To draw Sierpinski's triangle, we are going to use an external package, [_Compose.jl_](https://giovineitalia.github.io/Compose.jl/latest/tutorial). Let's set up a package environment and add the package. + +A package contains a coherent set of functionality that you can often use a black box according to its specification. There are [lots of Julia packages](https://juliahub.com/ui/Home). +" + +# ╔═╡ 851c03a4-e7a4-11ea-1652-d59b7a6599f0 +# setting up an empty package environment +begin + import Pkg + Pkg.activate(mktempdir()) + Pkg.Registry.update() +end + +# ╔═╡ d6ee91ea-e750-11ea-1260-31ebf3ec6a9b +# add (ie install) a package to our environment +begin + Pkg.add("Compose") + # call `using` so that we can use it in our code + using Compose +end + +# ╔═╡ 5acd58e0-e856-11ea-2d3d-8329889fe16f +begin + Pkg.add("PlutoUI") + using PlutoUI +end + +# ╔═╡ dbc4da6a-e7b4-11ea-3b70-6f2abfcab992 +md"Just like the definition above, our `sierpinksi` function is _recursive_: it calls itself." + +# ╔═╡ 02b9c9d6-e752-11ea-0f32-91b7b6481684 +complexity = 5 + +# ╔═╡ 1eb79812-e7b5-11ea-1c10-63b24803dd8a +if complexity == 3 + md""" +Try changing the value of **`complexity` to `5`** in the cell above. + +Hit `Shift+Enter` to affect the change. + """ +else + md""" +**Great!** As you can see, all the cells in this notebook are linked together by the variables they define and use. Just like a spreadsheet! + """ +end + +# ╔═╡ d7e8202c-e7b5-11ea-30d3-adcd6867d5f5 +md"### Exercise 2.1 + +As you can see, the total area covered by triangles is lower when the complexity is higher." + +# ╔═╡ f22222b4-e7b5-11ea-0ea0-8fa368d2a014 +md""" +Can you write a function that computes the _area of `sierpinski(n)`_, as a fraction of the area of `sierpinski(0)`? + +So: +``` +area_sierpinski(0) = 1.0 +area_sierpinski(1) = 0.?? +... +``` +""" + +# ╔═╡ ca8d2f72-e7b6-11ea-1893-f1e6d0a20dc7 +function area_sierpinski(n) + if (n == 0) + return 1.0 + end + return 0.75 * area_sierpinski(n-1) +end + +# ╔═╡ 71c78614-e7bc-11ea-0959-c7a91a10d481 +if area_sierpinski(0) == 1.0 && area_sierpinski(1) == 3 / 4 + md""" +!!! correct + Well done! + """ +else + md""" +!!! warning "Incorrect" + Keep working on it! + """ +end + +# ╔═╡ c21096c0-e856-11ea-3dc5-a5b0cbf29335 +md"**Let's try it out below:**" + +# ╔═╡ 52533e00-e856-11ea-08a7-25e556fb1127 +md"Complexity = $(@bind n Slider(0:6, show_value=true))" + +# ╔═╡ c1ecad86-e7bc-11ea-1201-23ee380181a1 +md""" +!!! hint + Can you write `area_sierpinksi(n)` as a function of `area_sierpinski(n-1)`? +""" + +# ╔═╡ c9bf4288-e6ce-11ea-0e13-a36b5e685998 + + +# ╔═╡ a60a492a-e7bc-11ea-0f0b-75d81ce46a01 +md"That's it for now, see you next week!" + +# ╔═╡ b3c7a050-e855-11ea-3a22-3f514da746a4 +if student.kerberos_id === "jazz" + md""" +!!! danger "Oops!" + **Before you submit**, remember to fill in your name and kerberos ID at the top of this notebook! + """ +end + +# ╔═╡ d3625d20-e6ce-11ea-394a-53208540d626 + + +# ╔═╡ dfdeab34-e751-11ea-0f90-2fa9bbdccb1e +triangle() = compose(context(), polygon([(1, 1), (0, 1), (1 / 2, 0)])) + +# ╔═╡ b923d394-e750-11ea-1971-595e09ab35b5 +# It does not matter which order you define the building blocks (functions) of the +# program in. The best way to organize code is the one that promotes understanding. + +function place_in_3_corners(t) + # Uses the Compose library to place 3 copies of t + # in the 3 corners of a triangle. + # treat this function as a black box, + # or learn how it works from the Compose documentation here https://giovineitalia.github.io/Compose.jl/latest/tutorial/#Compose-is-declarative-1 + compose(context(), + (context(1 / 4, 0, 1 / 2, 1 / 2), t), + (context(0, 1 / 2, 1 / 2, 1 / 2), t), + (context(1 / 2, 1 / 2, 1 / 2, 1 / 2), t)) +end + +# ╔═╡ e2848b9a-e703-11ea-24f9-b9131434a84b +function sierpinski(n) + if n == 0 + triangle() + else + t = sierpinski(n - 1) # recursively construct a smaller sierpinski's triangle + place_in_3_corners(t) # place it in the 3 corners of a triangle + end +end + +# ╔═╡ 9664ac52-e750-11ea-171c-e7d57741a68c +sierpinski(complexity) + +# ╔═╡ df0a4068-e7b2-11ea-2475-81b237d492b3 +sierpinski.(0:6) + +# ╔═╡ 147ed7b0-e856-11ea-0d0e-7ff0d527e352 +md""" + +Sierpinski's triangle of complexity $(n) + + $(sierpinski(n)) + +has area **$(area_sierpinski(n))** + +""" + +# ╔═╡ Cell order: +# ╟─fafae38e-e852-11ea-1208-732b4744e4c2 +# ╟─cdff6730-e785-11ea-2546-4969521b33a7 +# ╠═7308bc54-e6cd-11ea-0eab-83f7535edf25 +# ╟─a2181260-e6cd-11ea-2a69-8d9d31d1ef0e +# ╟─094e39c8-e6ce-11ea-131b-07c4a1199edf +# ╟─31a8fbf8-e6ce-11ea-2c66-4b4d02b41995 +# ╟─339c2d5c-e6ce-11ea-32f9-714b3628909c +# ╟─56866718-e6ce-11ea-0804-d108af4e5653 +# ╠═bccf0e88-e754-11ea-3ab8-0170c2d44628 +# ╟─e7abd366-e7a6-11ea-30d7-1b6194614d0a +# ╟─d62f223c-e754-11ea-2470-e72a605a9d7e +# ╠═4896bf0c-e754-11ea-19dc-1380bb356ab6 +# ╠═7a01a508-e78a-11ea-11da-999d38785348 +# ╟─682db9f8-e7b1-11ea-3949-6b683ca8b47b +# ╟─088cc652-e7a8-11ea-0ca7-f744f6f3afdd +# ╟─c18dce7a-e7a7-11ea-0a1a-f944d46754e5 +# ╟─5e24d95c-e6ce-11ea-24be-bb19e1e14657 +# ╟─6b8883f6-e7b3-11ea-155e-6f62117e123b +# ╠═851c03a4-e7a4-11ea-1652-d59b7a6599f0 +# ╠═d6ee91ea-e750-11ea-1260-31ebf3ec6a9b +# ╠═5acd58e0-e856-11ea-2d3d-8329889fe16f +# ╟─dbc4da6a-e7b4-11ea-3b70-6f2abfcab992 +# ╠═e2848b9a-e703-11ea-24f9-b9131434a84b +# ╠═9664ac52-e750-11ea-171c-e7d57741a68c +# ╠═02b9c9d6-e752-11ea-0f32-91b7b6481684 +# ╟─1eb79812-e7b5-11ea-1c10-63b24803dd8a +# ╟─d7e8202c-e7b5-11ea-30d3-adcd6867d5f5 +# ╠═df0a4068-e7b2-11ea-2475-81b237d492b3 +# ╟─f22222b4-e7b5-11ea-0ea0-8fa368d2a014 +# ╠═ca8d2f72-e7b6-11ea-1893-f1e6d0a20dc7 +# ╠═71c78614-e7bc-11ea-0959-c7a91a10d481 +# ╟─c21096c0-e856-11ea-3dc5-a5b0cbf29335 +# ╟─52533e00-e856-11ea-08a7-25e556fb1127 +# ╟─147ed7b0-e856-11ea-0d0e-7ff0d527e352 +# ╟─c1ecad86-e7bc-11ea-1201-23ee380181a1 +# ╟─c9bf4288-e6ce-11ea-0e13-a36b5e685998 +# ╟─a60a492a-e7bc-11ea-0f0b-75d81ce46a01 +# ╟─b3c7a050-e855-11ea-3a22-3f514da746a4 +# ╟─d3625d20-e6ce-11ea-394a-53208540d626 +# ╠═dfdeab34-e751-11ea-0f90-2fa9bbdccb1e +# ╠═b923d394-e750-11ea-1971-595e09ab35b5 diff --git a/hw1.jl b/hw1.jl new file mode 100644 index 0000000..8d84bf0 --- /dev/null +++ b/hw1.jl @@ -0,0 +1,1640 @@ +### A Pluto.jl notebook ### +# v0.12.15 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ 14a34dd8-2c6b-11eb-0efe-a9af24fb87bf +using Images + +# ╔═╡ 6b30dc38-ed6b-11ea-10f3-ab3f121bf4b8 +begin + Pkg.add("PlutoUI") + using PlutoUI +end + +# ╔═╡ 83eb9ca0-ed68-11ea-0bc5-99a09c68f867 +md"_homework 1, version 4_" + +# ╔═╡ ac8ff080-ed61-11ea-3650-d9df06123e1f +md""" + +# **Homework 1** - _convolutions_ +`18.S191`, fall 2020 + +This notebook contains _built-in, live answer checks_! In some exercises you will see a coloured box, which runs a test case on your code, and provides feedback based on the result. Simply edit the code, run it, and the check runs again. + +_For MIT students:_ there will also be some additional (secret) test cases that will be run as part of the grading process, and we will look at your notebook and write comments. + +Feel free to ask questions! +""" + +# ╔═╡ 911ccbce-ed68-11ea-3606-0384e7580d7c +# edit the code below to set your name and kerberos ID (i.e. email without @mit.edu) + +student = (name = "Jazzy Doe", kerberos_id = "jazz") + +# press the ▶ button in the bottom right of this cell to run your edits +# or use Shift+Enter + +# you might need to wait until all other cells in this notebook have completed running. +# scroll down the page to see what's up + +# ╔═╡ 8ef13896-ed68-11ea-160b-3550eeabbd7d +md""" + +Submission by: **_$(student.name)_** ($(student.kerberos_id)@mit.edu) +""" + +# ╔═╡ 5f95e01a-ee0a-11ea-030c-9dba276aba92 +md"_Let's create a package environment:_" + +# ╔═╡ 67461396-ee0a-11ea-3679-f31d46baa9b4 +md"_We set up Images.jl again:_" + +# ╔═╡ 540ccfcc-ee0a-11ea-15dc-4f8120063397 +md""" +## **Exercise 1** - _Manipulating vectors (1D images)_ + +A `Vector` is a 1D array. We can think of that as a 1D image. + +""" + +# ╔═╡ 467856dc-eded-11ea-0f83-13d939021ef3 +example_vector = [0.5, 0.4, 0.3, 0.2, 0.1, 0.0, 0.7, 0.0, 0.7, 0.9] + +# ╔═╡ ad6a33b0-eded-11ea-324c-cfabfd658b56 +md"#### Exerise 1.1 +👉 Make a random vector `random_vect` of length 10 using the `rand` function. +" + +# ╔═╡ f51333a6-eded-11ea-34e6-bfbb3a69bcb0 +random_vect = rand(0.0:0.01:1.0, 10) # replace this with your code! + +# ╔═╡ cf738088-eded-11ea-2915-61735c2aa990 +md"👉 Make a function `mean` using a `for` loop, which computes the mean/average of a vector of numbers." + +# ╔═╡ 0ffa8354-edee-11ea-2883-9d5bfea4a236 +function mean(x) + m = 0.0 + for i in 1:length(x) + m += x[i] + end + return m / length(x) +end + +# ╔═╡ 1f104ce4-ee0e-11ea-2029-1d9c817175af +mean([1, 2, 3]) + +# ╔═╡ 1f229ca4-edee-11ea-2c56-bb00cc6ea53c +md"👉 Define `m` to be the mean of `random_vect`." + +# ╔═╡ 2a391708-edee-11ea-124e-d14698171b68 +m = mean(random_vect) + +# ╔═╡ e2863d4c-edef-11ea-1d67-332ddca03cc4 +md"""👉 Write a function `demean`, which takes a vector `x` and subtracts the mean from each value in `x`.""" + +# ╔═╡ ec5efe8c-edef-11ea-2c6f-afaaeb5bc50c +function demean(x) + m = mean(x) + return [xi - m for xi in x] +end + +# ╔═╡ 00070040-2c6d-11eb-20cc-43903baab117 +demean([1, 2, 3]) + +# ╔═╡ 29e10640-edf0-11ea-0398-17dbf4242de3 +md"Let's check that the mean of the `demean(random_vect)` is 0: + +_Due to floating-point round-off error it may *not* be *exactly* 0._" + +# ╔═╡ 6f67657e-ee1a-11ea-0c2f-3d567bcfa6ea +if ismissing(random_vect) + md""" + !!! info + The following cells error because `random_vect` is not yet defined. Have you done the first exercise? + """ +end + +# ╔═╡ 73ef1d50-edf0-11ea-343c-d71706874c82 +copy_of_random_vect = copy(random_vect); # in case demean modifies `x` + +# ╔═╡ 38155b5a-edf0-11ea-3e3f-7163da7433fb +mean(demean(copy_of_random_vect)) + +# ╔═╡ a5f8bafe-edf0-11ea-0da3-3330861ae43a +md""" +#### Exercise 1.2 + +👉 Generate a vector of 100 zeros. Change the center 20 elements to 1. +""" + +# ╔═╡ b6b65b94-edf0-11ea-3686-fbff0ff53d08 +function create_bar() + n = 100 + m = 20 + x = zeros(n) + x[n ÷ 2 - m ÷ 2:n ÷ 2 + m ÷ 2] .= 1 + return x +end + +# ╔═╡ 22f28dae-edf2-11ea-25b5-11c369ae1253 +md""" +#### Exercise 1.3 + +👉 Write a function that turns a `Vector` of `Vector`s into a `Matrix`. +""" + +# ╔═╡ 8c19fb72-ed6c-11ea-2728-3fa9219eddc4 +function vecvec_to_matrix(vecvec) + rows = length(vecvec) + cols = length(vecvec[1]) + m = Array{typeof(vecvec[1][1])}(undef, rows, cols) + for i in 1:length(vecvec) + m[i,:] = vecvec[i] + end + return m +end + +# ╔═╡ c4761a7e-edf2-11ea-1e75-118e73dadbed +vecvec_to_matrix([[1,2], [3,4]]) + +# ╔═╡ 393667ca-edf2-11ea-09c5-c5d292d5e896 +md""" + + +👉 Write a function that turns a `Matrix` into a`Vector` of `Vector`s . +""" + +# ╔═╡ 9f1c6d04-ed6c-11ea-007b-75e7e780703d +function matrix_to_vecvec(matrix) + vv = Array{Array{typeof(matrix[1,1])}}(undef, size(matrix, 1)) + for i in 1:size(matrix,1) + vv[i] = matrix[i,:] + end + return vv +end + +# ╔═╡ 70955aca-ed6e-11ea-2330-89b4d20b1795 +matrix_to_vecvec([6 7; 8 9]) + +# ╔═╡ 5da8cbe8-eded-11ea-2e43-c5b7cc71e133 +begin + colored_line(x::Vector{<:Real}) = Gray.(Float64.((hcat(x)'))) + colored_line(x::Any) = nothing +end + +# ╔═╡ 56ced344-eded-11ea-3e81-3936e9ad5777 +colored_line(example_vector) + +# ╔═╡ b18e2c54-edf1-11ea-0cbf-85946d64b6a2 +colored_line(random_vect) + +# ╔═╡ d862fb16-edf1-11ea-36ec-615d521e6bc0 +colored_line(create_bar()) + +# ╔═╡ e083b3e8-ed61-11ea-2ec9-217820b0a1b4 +md""" +## **Exercise 2** - _Manipulating images_ + +In this exercise we will get familiar with matrices (2D arrays) in Julia, by manipulating images. +Recall that in Julia images are matrices of `RGB` color objects. + +Let's load a picture of Philip again. +""" + +# ╔═╡ c5484572-ee05-11ea-0424-f37295c3072d +philip_file = download("https://i.imgur.com/VGPeJ6s.jpg") + +# ╔═╡ e86ed944-ee05-11ea-3e0f-d70fc73b789c +md"_Hi there Philip_" + +# ╔═╡ c54ccdea-ee05-11ea-0365-23aaf053b7d7 +md""" +#### Exercise 2.1 +👉 Write a function **`mean_colors`** that accepts an object called `image`. It should calculate the mean (average) amounts of red, green and blue in the image and return a tuple `(r, g, b)` of those means. +""" + +# ╔═╡ f6898df6-ee07-11ea-2838-fde9bc739c11 +function mean_colors(image) + rgb_mean = RGB(0.0, 0.0, 0.0) + for i = 1:size(image,1), j = 1:size(image,2) + rgb_mean += image[i,j] + end + rgb_mean /= length(image) + return rgb_mean +end + +# ╔═╡ d75ec078-ee0d-11ea-3723-71fb8eecb040 + + +# ╔═╡ f68d4a36-ee07-11ea-0832-0360530f102e +md""" +#### Exercise 2.2 +👉 Look up the documentation on the `floor` function. Use it to write a function `quantize(x::Number)` that takes in a value $x$ (which you can assume is between 0 and 1) and "quantizes" it into bins of width 0.1. For example, check that 0.267 gets mapped to 0.2. +""" + +# ╔═╡ f6991a50-ee07-11ea-0bc4-1d68eb028e6a +begin + function quantize(x::Number) + return floor(10*x)/10 + end + + function quantize(color::AbstractRGB) + # you will write me in a later exercise! + return RGB(quantize(color.r), quantize(color.r), quantize(color.r)) + end + + function quantize(image::AbstractMatrix) + # you will write me in a later exercise! + m = similar(image) + quantize.(m) + #for i=1:size(image, 1), j=1:size(image,2) + # m[i,j] = quantize(image[i,j]) + #end + return m + end +end + +# ╔═╡ f6a655f8-ee07-11ea-13b6-43ca404ddfc7 +quantize(0.267), quantize(0.91) + +# ╔═╡ f6b218c0-ee07-11ea-2adb-1968c4fd473a +md""" +#### Exercise 2.3 +👉 Write the second **method** of the function `quantize`, i.e. a new *version* of the function with the *same* name. This method will accept a color object called `color`, of the type `AbstractRGB`. + +_Write the function in the same cell as `quantize(x::Number)` from the last exercise. 👆_ + +Here, `::AbstractRGB` is a **type annotation**. This ensures that this version of the function will be chosen when passing in an object whose type is a **subtype** of the `AbstractRGB` abstract type. For example, both the `RGB` and `RGBX` types satisfy this. + +The method you write should return a new `RGB` object, in which each component ($r$, $g$ and $b$) are quantized. +""" + +# ╔═╡ c00fb9b2-2c71-11eb-1a45-d11e4c4b7c37 +quantize(RGB(.23345, .39343, 0.40001)) + +# ╔═╡ f6bf64da-ee07-11ea-3efb-05af01b14f67 +md""" +#### Exercise 2.4 +👉 Write a method `quantize(image::AbstractMatrix)` that quantizes an image by quantizing each pixel in the image. (You may assume that the matrix is a matrix of color objects.) + +_Write the function in the same cell as `quantize(x::Number)` from the last exercise. 👆_ +""" + +# ╔═╡ 25dad7ce-ee0b-11ea-3e20-5f3019dd7fa3 +md"Let's apply your method!" + +# ╔═╡ f6cc03a0-ee07-11ea-17d8-013991514d42 +md""" +#### Exercise 2.5 +👉 Write a function `invert` that inverts a color, i.e. sends $(r, g, b)$ to $(1 - r, 1-g, 1-b)$. +""" + +# ╔═╡ 63e8d636-ee0b-11ea-173d-bd3327347d55 +function invert(color::AbstractRGB) + return RGB(1-color.r, 1-color.g, 1-color.b) +end + +# ╔═╡ 2cc2f84e-ee0d-11ea-373b-e7ad3204bb00 +md"Let's invert some colors:" + +# ╔═╡ b8f26960-ee0a-11ea-05b9-3f4bc1099050 +black = RGB(0.0, 0.0, 0.0) + +# ╔═╡ 5de3a22e-ee0b-11ea-230f-35df4ca3c96d +invert(black) + +# ╔═╡ 4e21e0c4-ee0b-11ea-3d65-b311ae3f98e9 +red = RGB(0.8, 0.1, 0.1) + +# ╔═╡ 6dbf67ce-ee0b-11ea-3b71-abc05a64dc43 +invert(red) + +# ╔═╡ 846b1330-ee0b-11ea-3579-7d90fafd7290 +md"Can you invert the picture of Philip?" + +# ╔═╡ f6d6c71a-ee07-11ea-2b63-d759af80707b +md""" +#### Exercise 2.6 +👉 Write a function `noisify(x::Number, s)` to add randomness of intensity $s$ to a value $x$, i.e. to add a random value between $-s$ and $+s$ to $x$. If the result falls outside the range $(0, 1)$ you should "clamp" it to that range. (Note that Julia has a `clamp` function, but you should write your own function `myclamp(x)`.) +""" + +# ╔═╡ f6e2cb2a-ee07-11ea-06ee-1b77e34c1e91 +begin + function myclamp(x::Number, min=0, max=1) + if (x < min) + return min + end + if (x > max) + return max + end + return x + end + + function myclamp(c::AbstractRGB) + return RGB(myclamp(c.r), myclamp(c.g), myclamp(c.b)) + end + + function myclamp(g::AbstractGray) + return Gray(myclamp(g.val)) + end + + function noisify(x::Number, s) + return myclamp(x + s - rand() * 2 * s) + end + + function noisify(color::AbstractRGB, s) + # you will write me in a later exercise! + return RGB(noisify(color.r, s), noisify(color.g, s), noisify(color.b, s)) + end + + function noisify(image::AbstractMatrix, s) + # you will write me in a later exercise! + return noisify.(image, s) + end +end + +# ╔═╡ 472c5136-2c73-11eb-30a2-49c7fbdd4d81 +noisify(0.3, 0.8) + +# ╔═╡ f6fc1312-ee07-11ea-39a0-299b67aee3d8 +md""" +👉 Write the second method `noisify(c::AbstractRGB, s)` to add random noise of intensity $s$ to each of the $(r, g, b)$ values in a colour. + +_Write the function in the same cell as `noisify(x::Number)` from the last exercise. 👆_ +""" + +# ╔═╡ 774b4ce6-ee1b-11ea-2b48-e38ee25fc89b +@bind color_noise Slider(0:0.01:1, show_value=true) + +# ╔═╡ 7e4aeb70-ee1b-11ea-100f-1952ba66f80f +noisify(red, color_noise) + +# ╔═╡ 6a05f568-ee1b-11ea-3b6c-83b6ada3680f + + +# ╔═╡ f70823d2-ee07-11ea-2bb3-01425212aaf9 +md""" +👉 Write the third method `noisify(image::AbstractMatrix, s)` to noisify each pixel of an image. + +_Write the function in the same cell as `noisify(x::Number)` from the last exercise. 👆_ +""" + +# ╔═╡ e70a84d4-ee0c-11ea-0640-bf78653ba102 +@bind philip_noise Slider(0:0.01:8, show_value=true) + +# ╔═╡ 9604bc44-ee1b-11ea-28f8-7f7af8d0cbb2 + + +# ╔═╡ f714699e-ee07-11ea-08b6-5f5169861b57 +md""" +👉 For which noise intensity does it become unrecognisable? + +You may need noise intensities larger than 1. Why? + +""" + +# ╔═╡ bdc2df7c-ee0c-11ea-2e9f-7d2c085617c1 +answer_about_noise_intensity = md""" +The image is unrecognisable with intensity ... +""" + +# ╔═╡ 81510a30-ee0e-11ea-0062-8b3327428f9d + + +# ╔═╡ e3b03628-ee05-11ea-23b6-27c7b0210532 +decimate(image, ratio=5) = image[1:ratio:end, 1:ratio:end] + +# ╔═╡ c8ecfe5c-ee05-11ea-322b-4b2714898831 +philip = let + original = Images.load(philip_file) + decimate(original, 8) +end + +# ╔═╡ 5be9b144-ee0d-11ea-2a8d-8775de265a1d +mean_colors(philip) + +# ╔═╡ 9751586e-ee0c-11ea-0cbb-b7eda92977c9 +quantize(philip) + +# ╔═╡ 943103e2-ee0b-11ea-33aa-75a8a1529931 +philip_inverted = invert.(philip) + +# ╔═╡ ac15e0d0-ee0c-11ea-1eaf-d7f88b5df1d7 +noisify(philip, philip_noise) + +# ╔═╡ e08781fa-ed61-11ea-13ae-91a49b5eb74a +md""" + +## **Exercise 3** - _Convolutions_ + +As we have seen in the videos, we can produce cool effects using the mathematical technique of **convolutions**. We input one image $M$ and get a new image $M'$ back. + +Conceptually we think of $M$ as a matrix. In practice, in Julia it will be a `Matrix` of color objects, and we may need to take that into account. Ideally, however, we should write a **generic** function that will work for any type of data contained in the matrix. + +A convolution works on a small **window** of an image, i.e. a region centered around a given point $(i, j)$. We will suppose that the window is a square region with odd side length $2\ell + 1$, running from $-\ell, \ldots, 0, \ldots, \ell$. + +The result of the convolution over a given window, centred at the point $(i, j)$ is a *single number*; this number is the value that we will use for $M'_{i, j}$. +(Note that neighbouring windows overlap.) + +To get started let's restrict ourselves to convolutions in 1D. +So a window is just a 1D region from $-\ell$ to $\ell$. + +""" + +# ╔═╡ 7fc8ee1c-ee09-11ea-1382-ad21d5373308 +md""" +--- + +Let's create a vector `v` of random numbers of length `n=100`. +""" + +# ╔═╡ 7fcd6230-ee09-11ea-314f-a542d00d582e +n = 30 + +# ╔═╡ 7fdb34dc-ee09-11ea-366b-ffe10d1aa845 +v = rand(n) + +# ╔═╡ 7fe9153e-ee09-11ea-15b3-6f24fcc20734 +md"_Feel free to experiment with different values!_" + +# ╔═╡ 80108d80-ee09-11ea-0368-31546eb0d3cc +md""" +#### Exercise 3.1 +You've seen some colored lines in this notebook to visualize arrays. Can you make another one? + +👉 Try plotting our vector `v` using `colored_line(v)`. +""" + +# ╔═╡ 01070e28-ee0f-11ea-1928-a7919d452bdd +colored_line(v) + +# ╔═╡ 7522f81e-ee1c-11ea-35af-a17eb257ff1a +md"Try changing `n` and `v` around. Notice that you can run the cell `v = rand(n)` again to regenerate new random values." + +# ╔═╡ 801d90c0-ee09-11ea-28d6-61b806de26dc +md""" +#### Exercise 3.2 +We need to decide how to handle the **boundary conditions**, i.e. what happens if we try to access a position in the vector `v` beyond `1:n`. The simplest solution is to assume that $v_{i}$ is 0 outside the original vector; however, this may lead to strange boundary effects. + +A better solution is to use the *closest* value that is inside the vector. Effectively we are extending the vector and copying the extreme values into the extended positions. (Indeed, this is one way we could implement this; these extra positions are called **ghost cells**.) + +👉 Write a function `extend(v, i)` that checks whether the position $i$ is inside `1:n`. If so, return the $i$th component of `v`; otherwise, return the nearest end value. +""" + +# ╔═╡ 802bec56-ee09-11ea-043e-51cf1db02a34 +function extend(v, i) + if (i < 1) + return v[1] + end + if (i > length(v)) + return v[end] + end + return v[i] +end + +# ╔═╡ b7f3994c-ee1b-11ea-211a-d144db8eafc2 +md"_Some test cases:_" + +# ╔═╡ 803905b2-ee09-11ea-2d52-e77ff79693b0 +extend(v, 1) + +# ╔═╡ 80479d98-ee09-11ea-169e-d166eef65874 +extend(v, -8) + +# ╔═╡ 805691ce-ee09-11ea-053d-6d2e299ee123 +extend(v, n + 10) + +# ╔═╡ 806e5766-ee0f-11ea-1efc-d753cd83d086 +md"Extended with 0:" + +# ╔═╡ 38da843a-ee0f-11ea-01df-bfa8b1317d36 +colored_line([0, 0, example_vector..., 0, 0]) + +# ╔═╡ 9bde9f92-ee0f-11ea-27f8-ffef5fce2b3c +md"Extended with your `extend`:" + +# ╔═╡ 45c4da9a-ee0f-11ea-2c5b-1f6704559137 +if extend(v,1) === missing + missing +else + colored_line([extend(example_vector, i) for i in -1:12]) +end + +# ╔═╡ 80664e8c-ee09-11ea-0702-711bce271315 +md""" +#### Exercise 3.3 +👉 Write a function `blur_1D(v, l)` that blurs a vector `v` with a window of length `l` by averaging the elements within a window from $-\ell$ to $\ell$. This is called a **box blur**. +""" + +# ╔═╡ 807e5662-ee09-11ea-3005-21fdcc36b023 +function blur_1D(v, l) + vout = similar(v) + box_size = 2*l+1 + for i=1:length(v) + vout[i] = sum([extend(v, i+offset) for offset in -l:l]) / box_size + end + return vout +end + +# ╔═╡ 808deca8-ee09-11ea-0ee3-1586fa1ce282 +let + try + test_v = rand(n) + original = copy(test_v) + blur_1D(test_v, 5) + if test_v != original + md""" + !!! danger "Oopsie!" + It looks like your function _modifies_ `v`. Can you write it without doing so? Maybe you can use `copy`. + """ + end + catch + end +end + +# ╔═╡ 809f5330-ee09-11ea-0e5b-415044b6ac1f +md""" +#### Exercise 3.4 +👉 Apply the box blur to your vector `v`. Show the original and the new vector by creating two cells that call `colored_line`. Make the parameter $\ell$ interactive, and call it `l_box` instead of just `l` to avoid a variable naming conflict. +""" + +# ╔═╡ ca1ac5f4-ee1c-11ea-3d00-ff5268866f87 +colored_line(v) + +# ╔═╡ bb1b5446-2c75-11eb-2158-ed77180b8d0e +@bind l_box Slider(0:length(v), show_value=true) + +# ╔═╡ a4eec7ca-2c75-11eb-3c3f-91a5cbf4b169 +colored_line(blur_1D(v, l_box)) + +# ╔═╡ 80ab64f4-ee09-11ea-29b4-498112ed0799 +md""" +#### Exercise 3.5 +The box blur is a simple example of a **convolution**, i.e. a linear function of a window around each point, given by + +$$v'_{i} = \sum_{n} \, v_{i - n} \, k_{n},$$ + +where $k$ is a vector called a **kernel**. + +Again, we need to take care about what happens if $v_{i -n }$ falls off the end of the vector. + +👉 Write a function `convolve_vector(v, k)` that performs this convolution. You need to think of the vector $k$ as being *centred* on the position $i$. So $n$ in the above formula runs between $-\ell$ and $\ell$, where $2\ell + 1$ is the length of the vector $k$. You will need to do the necessary manipulation of indices. +""" + +# ╔═╡ 28e20950-ee0c-11ea-0e0a-b5f2e570b56e +function convolve_vector(v, k) + vout = similar(v) + box_size = length(k) + l = (box_size - 1) ÷ 2 + for i=1:length(v) + vout[i] = sum([extend(v, i+offset)*k[1+l+offset] for offset in -l:l]) + end + return vout +end + +# ╔═╡ 93284f92-ee12-11ea-0342-833b1a30625c +test_convolution = let + v = [1, 10, 100, 1000, 10000] + k = [0, 1, 0] + convolve_vector(v, k) +end + +# ╔═╡ 5eea882c-ee13-11ea-0d56-af81ecd30a4a +colored_line(test_convolution) + +# ╔═╡ cf73f9f8-ee12-11ea-39ae-0107e9107ef5 +md"_Edit the cell above, or create a new cell with your own test cases!_" + +# ╔═╡ 80b7566a-ee09-11ea-3939-6fab470f9ec8 +md""" +#### Exercise 3.6 +👉 Write a function `gaussian_kernel`. + +The definition of a Gaussian in 1D is + +$$G(x) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp \left( \frac{-x^2}{2\sigma^2} \right)$$ + +We need to **sample** (i.e. evaluate) this at each pixel in a region of size $n^2$, +and then **normalize** so that the sum of the resulting kernel is 1. + +For simplicity you can take $\sigma=1$. +""" + +# ╔═╡ a20aac70-2c77-11eb-0081-c5683fc1aa20 +G(x) = exp(-x^2 / 2) / sqrt(2*pi) + +# ╔═╡ f8bd22b8-ee14-11ea-04aa-ab16fd01826e +md"Let's test your kernel function!" + +# ╔═╡ 2a9dd06a-ee13-11ea-3f84-67bb309c77a8 +gaussian_kernel_size_1D = 5 # change this value, or turn me into a slider! + +# ╔═╡ b01858b6-edf3-11ea-0826-938d33c19a43 +md""" + + +## **Exercise 4** - _Convolutions of images_ + +Now let's move to 2D images. The convolution is then given by a **kernel** matrix $K$: + +$$M'_{i, j} = \sum_{k, l} \, M_{i- k, j - l} \, K_{k, l},$$ + +where the sum is over the possible values of $k$ and $l$ in the window. Again we think of the window as being *centered* at $(i, j)$. + +A common notation for this operation is $*$: + +$$M' = M * K.$$ +""" + +# ╔═╡ 7c1bc062-ee15-11ea-30b1-1b1e76520f13 +md""" +#### Exercise 4.1 +👉 Write a function `extend_mat` that takes a matrix `M` and indices `i` and `j`, and returns the closest element of the matrix. +""" + +# ╔═╡ 7c2ec6c6-ee15-11ea-2d7d-0d9401a5e5d1 +function extend_mat(M::AbstractMatrix, i, j) + rows, cols = size(M) + if (i < 1) + i = 1 + end + if (i > rows) + i = rows + end + if (j < 1) + j = 1 + end + if (j > cols) + j = cols + end + return M[i, j] +end + +# ╔═╡ 9afc4dca-ee16-11ea-354f-1d827aaa61d2 +md"_Let's test it!_" + +# ╔═╡ cf6b05e2-ee16-11ea-3317-8919565cb56e +small_image = Gray.(rand(5,5)) + +# ╔═╡ e3616062-ee27-11ea-04a9-b9ec60842a64 +md"Extended with `0`:" + +# ╔═╡ e5b6cd34-ee27-11ea-0d60-bd4796540b18 +[get(small_image, (i, j), Gray(0)) for (i,j) in Iterators.product(-1:7,-1:7)] + +# ╔═╡ d06ea762-ee27-11ea-2e9c-1bcff86a3fe0 +md"Extended with your `extend`:" + +# ╔═╡ e1dc0622-ee16-11ea-274a-3b6ec9e15ab5 +[extend_mat(small_image, i, j) for (i,j) in Iterators.product(-1:7,-1:7)] + +# ╔═╡ 3cd535e4-ee26-11ea-2482-fb4ad43dda19 +let + philip_head = philip[250:430,110:230] + [extend_mat(philip_head, i, j) for (i,j) in Iterators.product(-50:size(philip_head,1)+51, (-50:size(philip_head,2)+51))] +end + +# ╔═╡ 7c41f0ca-ee15-11ea-05fb-d97a836659af +md""" +#### Exercise 4.2 +👉 Implement a function `convolve_image(M, K)`. +""" + +# ╔═╡ 8b96e0bc-ee15-11ea-11cd-cfecea7075a0 +function convolve_image(M::AbstractMatrix, K::AbstractMatrix) + Mout = similar(M) + rows, cols = size(M) + krows, kcols = size(K) + l = (krows-1) ÷ 2 + for i=1:rows, j=1:cols + Mout[i,j] = myclamp(sum([extend_mat(M, i+di, j+dj) * K[1+l+di, 1+l+dj] + for di in -l:l, dj in -l:l])) + end + return Mout +end + +# ╔═╡ 5a5135c6-ee1e-11ea-05dc-eb0c683c2ce5 +md"_Let's test it out! 🎃_" + +# ╔═╡ 577c6daa-ee1e-11ea-1275-b7abc7a27d73 +test_image_with_border = [get(small_image, (i, j), Gray(0)) for (i,j) in Iterators.product(-1:7,-1:7)] + +# ╔═╡ 275a99c8-ee1e-11ea-0a76-93e3618c9588 +K_test = [ + -1 0 1 + -2 0 2 + -1 0 1 +] + +# ╔═╡ 42dfa206-ee1e-11ea-1fcd-21671042064c +convolve_image(test_image_with_border, K_test) + +# ╔═╡ 6e53c2e6-ee1e-11ea-21bd-c9c05381be07 +md"_Edit_ `K_test` _to create your own test case!_" + +# ╔═╡ e7f8b41a-ee25-11ea-287a-e75d33fbd98b +convolve_image(philip, K_test) + +# ╔═╡ 8a335044-ee19-11ea-0255-b9391246d231 +md""" +--- + +You can create all sorts of effects by choosing the kernel in a smart way. Today, we will implement two special kernels, to produce a **Gaussian blur** and a **Sobel edge detect** filter. + +Make sure that you have watched [the lecture](https://www.youtube.com/watch?v=8rrHTtUzyZA) about convolutions! +""" + +# ╔═╡ 7c50ea80-ee15-11ea-328f-6b4e4ff20b7e +md""" +#### Exercise 4.3 +👉 Apply a **Gaussian blur** to an image. + +Here, the 2D Gaussian kernel will be defined as + +$$G(x,y)=\frac{1}{2\pi \sigma^2}e^{\frac{-(x^2+y^2)}{2\sigma^2}}$$ +""" + +# ╔═╡ 45901d9c-2c7a-11eb-0c67-9120fa6b35d9 +G(x,y) = exp(-(x^2+y^2)/2) / sqrt(2*pi) + +# ╔═╡ 1c8b4658-ee0c-11ea-2ede-9b9ed7d3125e +function gaussian_kernel(n) + n2 = n^2 + l = (n2-1) ÷ 2 + k = [G(x-l) for x in 1:n2] + return k ./ sum(k) +end + +# ╔═╡ 38eb92f6-ee13-11ea-14d7-a503ac04302e +test_gauss_1D_a = let + v = random_vect + k = gaussian_kernel(gaussian_kernel_size_1D) + + if k !== missing + convolve_vector(v, k) + end +end + +# ╔═╡ b424e2aa-ee14-11ea-33fa-35491e0b9c9d +colored_line(test_gauss_1D_a) + +# ╔═╡ 24c21c7c-ee14-11ea-1512-677980db1288 +test_gauss_1D_b = let + v = create_bar() + k = gaussian_kernel(gaussian_kernel_size_1D) + + if k !== missing + convolve_vector(v, k) + end +end + +# ╔═╡ bc1c20a4-ee14-11ea-3525-63c9fa78f089 +colored_line(test_gauss_1D_b) + +# ╔═╡ 39b6a9fe-2c7b-11eb-3fd1-53ee4a726633 +function gaussian_2d_kernel(n) + l = (n-1) ÷ 2 + K = [G(x, y) for x in -l:l, y in -l:l] + return K ./ sum(K) +end + +# ╔═╡ a4477dc0-2c7b-11eb-0ca7-7f796edf4434 +gaussian_2d_kernel(5) + +# ╔═╡ aad67fd0-ee15-11ea-00d4-274ec3cda3a3 +function with_gaussian_blur(image, n=3) + K = gaussian_2d_kernel(n) + return convolve_image(image, K) +end + +# ╔═╡ 8ae59674-ee18-11ea-3815-f50713d0fa08 +md"_Let's make it interactive. 💫_" + +# ╔═╡ a75701c4-ee18-11ea-2863-d3042e71a68b +with_gaussian_blur(philip, 5) + +# ╔═╡ 7c6642a6-ee15-11ea-0526-a1aac4286cdd +md""" +#### Exercise 4.4 +👉 Create a **Sobel edge detection filter**. + +Here, we will need to create two separate filters that separately detect edges in the horizontal and vertical directions: + +```math +\begin{align} + +G_x &= \left(\begin{bmatrix} +1 \\ +2 \\ +1 \\ +\end{bmatrix} \otimes [1~0~-1] +\right) * A = \begin{bmatrix} +1 & 0 & -1 \\ +2 & 0 & -2 \\ +1 & 0 & -1 \\ +\end{bmatrix}*A\\ +G_y &= \left( +\begin{bmatrix} +1 \\ +0 \\ +-1 \\ +\end{bmatrix} \otimes [1~2~1] +\right) * A = \begin{bmatrix} +1 & 2 & 1 \\ +0 & 0 & 0 \\ +-1 & -2 & -1 \\ +\end{bmatrix}*A +\end{align} +``` +Here $A$ is the array corresponding to your image. +We can think of these as derivatives in the $x$ and $y$ directions. + +Then we combine them by finding the magnitude of the **gradient** (in the sense of multivariate calculus) by defining + +$$G_\text{total} = \sqrt{G_x^2 + G_y^2}.$$ + +For simplicity you can choose one of the "channels" (colours) in the image to apply this to. +""" + +# ╔═╡ 9eeb876c-ee15-11ea-1794-d3ea79f47b75 +begin +function sq_c(c::AbstractRGB) + return RGB(c.r^2, c.g^2, c.b^2) +end + +function sqrt_c(c::AbstractRGB) + return RGB(sqrt(c.r), sqrt(c.g), sqrt(c.b)) +end + +function with_sobel_edge_detect(image) + Gx = [1 0 -1; 2 0 -2; 1 0 -1] + Gy = transpose(Gx) + xout = convolve_image(image, Gx) + yout = convolve_image(image, Gy) + return sqrt_c.(sq_c.(xout) + sq_c.(yout)) +end +end + +# ╔═╡ 1bf94c00-ee19-11ea-0e3c-e12bc68d8e28 +with_sobel_edge_detect(philip) + +# ╔═╡ 1b85ee76-ee10-11ea-36d7-978340ef61e6 +md""" +## **Exercise 5** - _Lecture transcript_ +_(MIT students only)_ + +Please see the Canvas post for transcript document for week 1 [here](https://canvas.mit.edu/courses/5637/discussion_topics/27880). + +We need each of you to correct about 100 lines (see instructions in the beginning of the document.) + +👉 Please mention the name of the video and the line ranges you edited: +""" + +# ╔═╡ 477d0a3c-ee10-11ea-11cf-07b0e0ce6818 +lines_i_edited = md""" +Convolution, lines 100-0 (_for example_) +""" + +# ╔═╡ 8ffe16ce-ee20-11ea-18bd-15640f94b839 +if student.kerberos_id === "jazz" + md""" +!!! danger "Oops!" + **Before you submit**, remember to fill in your name and kerberos ID at the top of this notebook! + """ +end + +# ╔═╡ 5516c800-edee-11ea-12cf-3f8c082ef0ef +hint(text) = Markdown.MD(Markdown.Admonition("hint", "Hint", [text])) + +# ╔═╡ b1d5ca28-edf6-11ea-269e-75a9fb549f1d +hint(md"You can find out more about any function (like `rand`) by creating a new cell and typing: + +``` +?rand +``` + +Once the Live Docs are open, you can select any code to learn more about it. It might be useful to leave it open all the time, and get documentation while you type code.") + +# ╔═╡ f6ef2c2e-ee07-11ea-13a8-2512e7d94426 +hint(md"The `rand` function generates (uniform) random floating-point numbers between $0$ and $1$.") + +# ╔═╡ ea435e58-ee11-11ea-3785-01af8dd72360 +hint(md"Have a look at Exercise 2 to see an example of adding interactivity with a slider. You can read the [Interactivity](./sample/Interactivity.jl) and the [PlutoUI](./sample/PlutoUI.jl) sample notebooks _(right click -> Open in new tab)_ to learn more.") + +# ╔═╡ e9aadeee-ee1d-11ea-3525-95f6ba5fda31 +hint(md"`l = (length(k) - 1) ÷ 2`") + +# ╔═╡ 649df270-ee24-11ea-397e-79c4355e38db +hint(md"`num_rows, num_columns = size(M)`") + +# ╔═╡ 0cabed84-ee1e-11ea-11c1-7d8a4b4ad1af +hint(md"`num_rows, num_columns = size(K)`") + +# ╔═╡ 57360a7a-edee-11ea-0c28-91463ece500d +almost(text) = Markdown.MD(Markdown.Admonition("warning", "Almost there!", [text])) + +# ╔═╡ dcb8324c-edee-11ea-17ff-375ff5078f43 +still_missing(text=md"Replace `missing` with your answer.") = Markdown.MD(Markdown.Admonition("warning", "Here we go!", [text])) + +# ╔═╡ 58af703c-edee-11ea-2963-f52e78fc2412 +keep_working(text=md"The answer is not quite right.") = Markdown.MD(Markdown.Admonition("danger", "Keep working on it!", [text])) + +# ╔═╡ f3d00a9a-edf3-11ea-07b3-1db5c6d0b3cf +yays = [md"Great!", md"Yay ❤", md"Great! 🎉", md"Well done!", md"Keep it up!", md"Good job!", md"Awesome!", md"You got the right answer!", md"Let's move on to the next section."] + +# ╔═╡ 5aa9dfb2-edee-11ea-3754-c368fb40637c +correct(text=rand(yays)) = Markdown.MD(Markdown.Admonition("correct", "Got it!", [text])) + +# ╔═╡ 74d44e22-edee-11ea-09a0-69aa0aba3281 +not_defined(variable_name) = Markdown.MD(Markdown.Admonition("danger", "Oopsie!", [md"Make sure that you define a variable called **$(Markdown.Code(string(variable_name)))**"])) + +# ╔═╡ 397941fc-edee-11ea-33f2-5d46c759fbf7 +if !@isdefined(random_vect) + not_defined(:random_vect) +elseif ismissing(random_vect) + still_missing() +elseif !(random_vect isa Vector) + keep_working(md"`random_vect` should be a `Vector`.") +elseif length(random_vect) != 10 + keep_working(md"`random_vect` does not have the correct size.") +else + correct() +end + +# ╔═╡ 38dc80a0-edef-11ea-10e9-615255a4588c +if !@isdefined(mean) + not_defined(:mean) +else + let + result = mean([1,2,3]) + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif result != 2 + keep_working() + else + correct() + end + end +end + +# ╔═╡ 2b1ccaca-edee-11ea-34b0-c51659f844d0 +if !@isdefined(m) + not_defined(:m) +elseif ismissing(m) + still_missing() +elseif !(m isa Number) + keep_working(md"`m` should be a number.") +elseif m != mean(random_vect) + keep_working() +else + correct() +end + +# ╔═╡ e3394c8a-edf0-11ea-1bb8-619f7abb6881 +if !@isdefined(create_bar) + not_defined(:create_bar) +else + let + result = create_bar() + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif !(result isa Vector) || length(result) != 100 + keep_working(md"The result should be a `Vector` with 100 elements.") + elseif result[[1,50,100]] != [0,1,0] + keep_working() + else + correct() + end + end +end + +# ╔═╡ adfbe9b2-ed6c-11ea-09ac-675262f420df +if !@isdefined(vecvec_to_matrix) + not_defined(:vecvec_to_matrix) +else + let + input = [[6,7],[8,9]] + + result = vecvec_to_matrix(input) + shouldbe = [6 7; 8 9] + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif !(result isa Matrix) + keep_working(md"The result should be a `Matrix`") + elseif result != shouldbe && result != shouldbe' + keep_working() + else + correct() + end + end +end + +# ╔═╡ e06b7fbc-edf2-11ea-1708-fb32599dded3 +if !@isdefined(matrix_to_vecvec) + not_defined(:matrix_to_vecvec) +else + let + input = [6 7 8; 8 9 10] + result = matrix_to_vecvec(input) + shouldbe = [[6,7,8],[8,9,10]] + shouldbe2 = [[6,8], [7,9], [8,10]] + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif result != shouldbe && result != shouldbe2 + keep_working() + else + correct() + end + end +end + +# ╔═╡ 4d0158d0-ee0d-11ea-17c3-c169d4284acb +if !@isdefined(mean_colors) + not_defined(:mean_colors) +else + let + input = reshape([RGB(1.0, 1.0, 1.0), RGB(1.0, 1.0, 0.0)], (2,1)) + + result = mean_colors(input) + shouldbe = (1.0, 1.0, 0.5) + shouldbe2 = RGB(shouldbe...) + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif !(result == shouldbe) && !(result == shouldbe2) + keep_working() + else + correct() + end + end +end + +# ╔═╡ c905b73e-ee1a-11ea-2e36-23b8e73bfdb6 +if !@isdefined(quantize) + not_defined(:quantize) +else + let + result = quantize(.3) + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif result != .3 + if quantize(0.35) == .3 + almost(md"What should quantize(`0.2`) be?") + else + keep_working() + end + else + correct() + end + end +end + +# ╔═╡ bcf98dfc-ee1b-11ea-21d0-c14439500971 +if !@isdefined(extend) + not_defined(:extend) +else + let + result = extend([6,7],-10) + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif result != 6 || extend([6,7],10) != 7 + keep_working() + else + correct() + end + end +end + +# ╔═╡ 7ffd14f8-ee1d-11ea-0343-b54fb0333aea +if !@isdefined(convolve_vector) + not_defined(:convolve_vector) +else + let + x = [1, 10, 100] + result = convolve_vector(x, [0, 1, 1]) + shouldbe = [11, 110, 200] + shouldbe2 = [2, 11, 110] + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif !(result isa AbstractVector) + keep_working(md"The returned object is not a `Vector`.") + elseif size(result) != size(x) + keep_working(md"The returned vector has the wrong dimensions.") + elseif result != shouldbe && result != shouldbe2 + keep_working() + else + correct() + end + end +end + +# ╔═╡ efd1ceb4-ee1c-11ea-350e-f7e3ea059024 +if !@isdefined(extend_mat) + not_defined(:extend_mat) +else + let + input = [42 37; 1 0] + result = extend_mat(input, -2, -2) + + if ismissing(result) + still_missing() + elseif isnothing(result) + keep_working(md"Did you forget to write `return`?") + elseif result != 42 || extend_mat(input, -1, 3) != 37 + keep_working() + else + correct() + end + end +end + +# ╔═╡ 115ded8c-ee0a-11ea-3493-89487315feb7 +bigbreak = html"




"; + +# ╔═╡ 45815734-ee0a-11ea-2982-595e1fc0e7b1 +bigbreak + +# ╔═╡ 4139ee66-ee0a-11ea-2282-15d63bcca8b8 +bigbreak + +# ╔═╡ 27847dc4-ee0a-11ea-0651-ebbbb3cfd58c +bigbreak + +# ╔═╡ 0001f782-ee0e-11ea-1fb4-2b5ef3d241e2 +bigbreak + +# ╔═╡ 91f4778e-ee20-11ea-1b7e-2b0892bd3c0f +bigbreak + +# ╔═╡ 5842895a-ee10-11ea-119d-81e4c4c8c53b +bigbreak + +# ╔═╡ dfb7c6be-ee0d-11ea-194e-9758857f7b20 +function camera_input(;max_size=200, default_url="https://i.imgur.com/SUmi94P.png") +""" + + + +
+
+ + +
+ +
+ +
+
+ +
+ + Enable webcam + +
+ + + +""" |> HTML +end + +# ╔═╡ 94c0798e-ee18-11ea-3212-1533753eabb6 +@bind gauss_raw_camera_data camera_input(;max_size=100) + +# ╔═╡ 1a0324de-ee19-11ea-1d4d-db37f4136ad3 +@bind sobel_raw_camera_data camera_input(;max_size=100) + +# ╔═╡ e15ad330-ee0d-11ea-25b6-1b1b3f3d7888 + +function process_raw_camera_data(raw_camera_data) + # the raw image data is a long byte array, we need to transform it into something + # more "Julian" - something with more _structure_. + + # The encoding of the raw byte stream is: + # every 4 bytes is a single pixel + # every pixel has 4 values: Red, Green, Blue, Alpha + # (we ignore alpha for this notebook) + + # So to get the red values for each pixel, we take every 4th value, starting at + # the 1st: + reds_flat = UInt8.(raw_camera_data["data"][1:4:end]) + greens_flat = UInt8.(raw_camera_data["data"][2:4:end]) + blues_flat = UInt8.(raw_camera_data["data"][3:4:end]) + + # but these are still 1-dimensional arrays, nicknamed 'flat' arrays + # We will 'reshape' this into 2D arrays: + + width = raw_camera_data["width"] + height = raw_camera_data["height"] + + # shuffle and flip to get it in the right shape + reds = reshape(reds_flat, (width, height))' / 255.0 + greens = reshape(greens_flat, (width, height))' / 255.0 + blues = reshape(blues_flat, (width, height))' / 255.0 + + # we have our 2D array for each color + # Let's create a single 2D array, where each value contains the R, G and B value of + # that pixel + + RGB.(reds, greens, blues) +end + +# ╔═╡ f461f5f2-ee18-11ea-3d03-95f57f9bf09e +gauss_camera_image = process_raw_camera_data(gauss_raw_camera_data); + +# ╔═╡ 1ff6b5cc-ee19-11ea-2ca8-7f00c204f587 +sobel_camera_image = Gray.(process_raw_camera_data(sobel_raw_camera_data)); + +# ╔═╡ Cell order: +# ╠═83eb9ca0-ed68-11ea-0bc5-99a09c68f867 +# ╟─8ef13896-ed68-11ea-160b-3550eeabbd7d +# ╟─ac8ff080-ed61-11ea-3650-d9df06123e1f +# ╠═911ccbce-ed68-11ea-3606-0384e7580d7c +# ╟─5f95e01a-ee0a-11ea-030c-9dba276aba92 +# ╟─67461396-ee0a-11ea-3679-f31d46baa9b4 +# ╠═14a34dd8-2c6b-11eb-0efe-a9af24fb87bf +# ╟─540ccfcc-ee0a-11ea-15dc-4f8120063397 +# ╟─467856dc-eded-11ea-0f83-13d939021ef3 +# ╠═56ced344-eded-11ea-3e81-3936e9ad5777 +# ╟─ad6a33b0-eded-11ea-324c-cfabfd658b56 +# ╠═f51333a6-eded-11ea-34e6-bfbb3a69bcb0 +# ╟─b18e2c54-edf1-11ea-0cbf-85946d64b6a2 +# ╟─397941fc-edee-11ea-33f2-5d46c759fbf7 +# ╟─b1d5ca28-edf6-11ea-269e-75a9fb549f1d +# ╟─cf738088-eded-11ea-2915-61735c2aa990 +# ╠═0ffa8354-edee-11ea-2883-9d5bfea4a236 +# ╠═1f104ce4-ee0e-11ea-2029-1d9c817175af +# ╟─38dc80a0-edef-11ea-10e9-615255a4588c +# ╟─1f229ca4-edee-11ea-2c56-bb00cc6ea53c +# ╠═2a391708-edee-11ea-124e-d14698171b68 +# ╟─2b1ccaca-edee-11ea-34b0-c51659f844d0 +# ╟─e2863d4c-edef-11ea-1d67-332ddca03cc4 +# ╠═ec5efe8c-edef-11ea-2c6f-afaaeb5bc50c +# ╠═00070040-2c6d-11eb-20cc-43903baab117 +# ╟─29e10640-edf0-11ea-0398-17dbf4242de3 +# ╟─6f67657e-ee1a-11ea-0c2f-3d567bcfa6ea +# ╠═38155b5a-edf0-11ea-3e3f-7163da7433fb +# ╠═73ef1d50-edf0-11ea-343c-d71706874c82 +# ╟─a5f8bafe-edf0-11ea-0da3-3330861ae43a +# ╠═b6b65b94-edf0-11ea-3686-fbff0ff53d08 +# ╟─d862fb16-edf1-11ea-36ec-615d521e6bc0 +# ╟─e3394c8a-edf0-11ea-1bb8-619f7abb6881 +# ╟─22f28dae-edf2-11ea-25b5-11c369ae1253 +# ╠═8c19fb72-ed6c-11ea-2728-3fa9219eddc4 +# ╠═c4761a7e-edf2-11ea-1e75-118e73dadbed +# ╟─adfbe9b2-ed6c-11ea-09ac-675262f420df +# ╟─393667ca-edf2-11ea-09c5-c5d292d5e896 +# ╠═9f1c6d04-ed6c-11ea-007b-75e7e780703d +# ╠═70955aca-ed6e-11ea-2330-89b4d20b1795 +# ╟─e06b7fbc-edf2-11ea-1708-fb32599dded3 +# ╟─5da8cbe8-eded-11ea-2e43-c5b7cc71e133 +# ╟─45815734-ee0a-11ea-2982-595e1fc0e7b1 +# ╟─e083b3e8-ed61-11ea-2ec9-217820b0a1b4 +# ╠═c5484572-ee05-11ea-0424-f37295c3072d +# ╠═c8ecfe5c-ee05-11ea-322b-4b2714898831 +# ╟─e86ed944-ee05-11ea-3e0f-d70fc73b789c +# ╟─c54ccdea-ee05-11ea-0365-23aaf053b7d7 +# ╠═f6898df6-ee07-11ea-2838-fde9bc739c11 +# ╠═5be9b144-ee0d-11ea-2a8d-8775de265a1d +# ╟─4d0158d0-ee0d-11ea-17c3-c169d4284acb +# ╠═d75ec078-ee0d-11ea-3723-71fb8eecb040 +# ╟─f68d4a36-ee07-11ea-0832-0360530f102e +# ╠═f6991a50-ee07-11ea-0bc4-1d68eb028e6a +# ╠═f6a655f8-ee07-11ea-13b6-43ca404ddfc7 +# ╟─c905b73e-ee1a-11ea-2e36-23b8e73bfdb6 +# ╟─f6b218c0-ee07-11ea-2adb-1968c4fd473a +# ╠═c00fb9b2-2c71-11eb-1a45-d11e4c4b7c37 +# ╟─f6bf64da-ee07-11ea-3efb-05af01b14f67 +# ╟─25dad7ce-ee0b-11ea-3e20-5f3019dd7fa3 +# ╠═9751586e-ee0c-11ea-0cbb-b7eda92977c9 +# ╟─f6cc03a0-ee07-11ea-17d8-013991514d42 +# ╠═63e8d636-ee0b-11ea-173d-bd3327347d55 +# ╟─2cc2f84e-ee0d-11ea-373b-e7ad3204bb00 +# ╟─b8f26960-ee0a-11ea-05b9-3f4bc1099050 +# ╠═5de3a22e-ee0b-11ea-230f-35df4ca3c96d +# ╠═4e21e0c4-ee0b-11ea-3d65-b311ae3f98e9 +# ╠═6dbf67ce-ee0b-11ea-3b71-abc05a64dc43 +# ╟─846b1330-ee0b-11ea-3579-7d90fafd7290 +# ╠═943103e2-ee0b-11ea-33aa-75a8a1529931 +# ╟─f6d6c71a-ee07-11ea-2b63-d759af80707b +# ╠═f6e2cb2a-ee07-11ea-06ee-1b77e34c1e91 +# ╠═472c5136-2c73-11eb-30a2-49c7fbdd4d81 +# ╟─f6ef2c2e-ee07-11ea-13a8-2512e7d94426 +# ╟─f6fc1312-ee07-11ea-39a0-299b67aee3d8 +# ╠═774b4ce6-ee1b-11ea-2b48-e38ee25fc89b +# ╠═7e4aeb70-ee1b-11ea-100f-1952ba66f80f +# ╟─6a05f568-ee1b-11ea-3b6c-83b6ada3680f +# ╟─f70823d2-ee07-11ea-2bb3-01425212aaf9 +# ╠═e70a84d4-ee0c-11ea-0640-bf78653ba102 +# ╠═ac15e0d0-ee0c-11ea-1eaf-d7f88b5df1d7 +# ╟─9604bc44-ee1b-11ea-28f8-7f7af8d0cbb2 +# ╟─f714699e-ee07-11ea-08b6-5f5169861b57 +# ╠═bdc2df7c-ee0c-11ea-2e9f-7d2c085617c1 +# ╟─81510a30-ee0e-11ea-0062-8b3327428f9d +# ╠═6b30dc38-ed6b-11ea-10f3-ab3f121bf4b8 +# ╟─e3b03628-ee05-11ea-23b6-27c7b0210532 +# ╟─4139ee66-ee0a-11ea-2282-15d63bcca8b8 +# ╟─e08781fa-ed61-11ea-13ae-91a49b5eb74a +# ╟─7fc8ee1c-ee09-11ea-1382-ad21d5373308 +# ╠═7fcd6230-ee09-11ea-314f-a542d00d582e +# ╠═7fdb34dc-ee09-11ea-366b-ffe10d1aa845 +# ╟─7fe9153e-ee09-11ea-15b3-6f24fcc20734 +# ╟─80108d80-ee09-11ea-0368-31546eb0d3cc +# ╠═01070e28-ee0f-11ea-1928-a7919d452bdd +# ╟─7522f81e-ee1c-11ea-35af-a17eb257ff1a +# ╟─801d90c0-ee09-11ea-28d6-61b806de26dc +# ╠═802bec56-ee09-11ea-043e-51cf1db02a34 +# ╟─b7f3994c-ee1b-11ea-211a-d144db8eafc2 +# ╠═803905b2-ee09-11ea-2d52-e77ff79693b0 +# ╠═80479d98-ee09-11ea-169e-d166eef65874 +# ╠═805691ce-ee09-11ea-053d-6d2e299ee123 +# ╟─806e5766-ee0f-11ea-1efc-d753cd83d086 +# ╟─38da843a-ee0f-11ea-01df-bfa8b1317d36 +# ╟─9bde9f92-ee0f-11ea-27f8-ffef5fce2b3c +# ╟─45c4da9a-ee0f-11ea-2c5b-1f6704559137 +# ╟─bcf98dfc-ee1b-11ea-21d0-c14439500971 +# ╟─80664e8c-ee09-11ea-0702-711bce271315 +# ╠═807e5662-ee09-11ea-3005-21fdcc36b023 +# ╟─808deca8-ee09-11ea-0ee3-1586fa1ce282 +# ╟─809f5330-ee09-11ea-0e5b-415044b6ac1f +# ╠═ca1ac5f4-ee1c-11ea-3d00-ff5268866f87 +# ╠═bb1b5446-2c75-11eb-2158-ed77180b8d0e +# ╠═a4eec7ca-2c75-11eb-3c3f-91a5cbf4b169 +# ╟─ea435e58-ee11-11ea-3785-01af8dd72360 +# ╟─80ab64f4-ee09-11ea-29b4-498112ed0799 +# ╠═28e20950-ee0c-11ea-0e0a-b5f2e570b56e +# ╟─e9aadeee-ee1d-11ea-3525-95f6ba5fda31 +# ╟─5eea882c-ee13-11ea-0d56-af81ecd30a4a +# ╠═93284f92-ee12-11ea-0342-833b1a30625c +# ╟─cf73f9f8-ee12-11ea-39ae-0107e9107ef5 +# ╟─7ffd14f8-ee1d-11ea-0343-b54fb0333aea +# ╟─80b7566a-ee09-11ea-3939-6fab470f9ec8 +# ╠═a20aac70-2c77-11eb-0081-c5683fc1aa20 +# ╠═1c8b4658-ee0c-11ea-2ede-9b9ed7d3125e +# ╟─f8bd22b8-ee14-11ea-04aa-ab16fd01826e +# ╠═2a9dd06a-ee13-11ea-3f84-67bb309c77a8 +# ╟─b424e2aa-ee14-11ea-33fa-35491e0b9c9d +# ╠═38eb92f6-ee13-11ea-14d7-a503ac04302e +# ╟─bc1c20a4-ee14-11ea-3525-63c9fa78f089 +# ╠═24c21c7c-ee14-11ea-1512-677980db1288 +# ╟─27847dc4-ee0a-11ea-0651-ebbbb3cfd58c +# ╟─b01858b6-edf3-11ea-0826-938d33c19a43 +# ╟─7c1bc062-ee15-11ea-30b1-1b1e76520f13 +# ╠═7c2ec6c6-ee15-11ea-2d7d-0d9401a5e5d1 +# ╟─649df270-ee24-11ea-397e-79c4355e38db +# ╟─9afc4dca-ee16-11ea-354f-1d827aaa61d2 +# ╠═cf6b05e2-ee16-11ea-3317-8919565cb56e +# ╟─e3616062-ee27-11ea-04a9-b9ec60842a64 +# ╟─e5b6cd34-ee27-11ea-0d60-bd4796540b18 +# ╟─d06ea762-ee27-11ea-2e9c-1bcff86a3fe0 +# ╟─e1dc0622-ee16-11ea-274a-3b6ec9e15ab5 +# ╟─efd1ceb4-ee1c-11ea-350e-f7e3ea059024 +# ╟─3cd535e4-ee26-11ea-2482-fb4ad43dda19 +# ╟─7c41f0ca-ee15-11ea-05fb-d97a836659af +# ╠═8b96e0bc-ee15-11ea-11cd-cfecea7075a0 +# ╟─0cabed84-ee1e-11ea-11c1-7d8a4b4ad1af +# ╟─5a5135c6-ee1e-11ea-05dc-eb0c683c2ce5 +# ╟─577c6daa-ee1e-11ea-1275-b7abc7a27d73 +# ╠═275a99c8-ee1e-11ea-0a76-93e3618c9588 +# ╠═42dfa206-ee1e-11ea-1fcd-21671042064c +# ╟─6e53c2e6-ee1e-11ea-21bd-c9c05381be07 +# ╠═e7f8b41a-ee25-11ea-287a-e75d33fbd98b +# ╟─8a335044-ee19-11ea-0255-b9391246d231 +# ╟─7c50ea80-ee15-11ea-328f-6b4e4ff20b7e +# ╠═45901d9c-2c7a-11eb-0c67-9120fa6b35d9 +# ╠═39b6a9fe-2c7b-11eb-3fd1-53ee4a726633 +# ╠═a4477dc0-2c7b-11eb-0ca7-7f796edf4434 +# ╠═aad67fd0-ee15-11ea-00d4-274ec3cda3a3 +# ╟─8ae59674-ee18-11ea-3815-f50713d0fa08 +# ╟─94c0798e-ee18-11ea-3212-1533753eabb6 +# ╠═a75701c4-ee18-11ea-2863-d3042e71a68b +# ╟─f461f5f2-ee18-11ea-3d03-95f57f9bf09e +# ╟─7c6642a6-ee15-11ea-0526-a1aac4286cdd +# ╠═9eeb876c-ee15-11ea-1794-d3ea79f47b75 +# ╟─1a0324de-ee19-11ea-1d4d-db37f4136ad3 +# ╠═1bf94c00-ee19-11ea-0e3c-e12bc68d8e28 +# ╟─1ff6b5cc-ee19-11ea-2ca8-7f00c204f587 +# ╟─0001f782-ee0e-11ea-1fb4-2b5ef3d241e2 +# ╠═1b85ee76-ee10-11ea-36d7-978340ef61e6 +# ╠═477d0a3c-ee10-11ea-11cf-07b0e0ce6818 +# ╟─91f4778e-ee20-11ea-1b7e-2b0892bd3c0f +# ╟─8ffe16ce-ee20-11ea-18bd-15640f94b839 +# ╟─5842895a-ee10-11ea-119d-81e4c4c8c53b +# ╟─5516c800-edee-11ea-12cf-3f8c082ef0ef +# ╟─57360a7a-edee-11ea-0c28-91463ece500d +# ╟─dcb8324c-edee-11ea-17ff-375ff5078f43 +# ╟─58af703c-edee-11ea-2963-f52e78fc2412 +# ╟─f3d00a9a-edf3-11ea-07b3-1db5c6d0b3cf +# ╟─5aa9dfb2-edee-11ea-3754-c368fb40637c +# ╟─74d44e22-edee-11ea-09a0-69aa0aba3281 +# ╟─115ded8c-ee0a-11ea-3493-89487315feb7 +# ╟─dfb7c6be-ee0d-11ea-194e-9758857f7b20 +# ╟─e15ad330-ee0d-11ea-25b6-1b1b3f3d7888 diff --git a/hw2.jl b/hw2.jl new file mode 100644 index 0000000..8dab36a --- /dev/null +++ b/hw2.jl @@ -0,0 +1,1035 @@ +### A Pluto.jl notebook ### +# v0.12.15 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ a4937996-f314-11ea-2ff9-615c888afaa8 +begin + using Images + using TestImages + using ImageFiltering + using Statistics + using PlutoUI + using BenchmarkTools +end + +# ╔═╡ e6b6760a-f37f-11ea-3ae1-65443ef5a81a +md"_homework 2, version 2.1_" + +# ╔═╡ 85cfbd10-f384-11ea-31dc-b5693630a4c5 +md""" + +# **Homework 2**: _Dynamic programming_ +`18.S191`, fall 2020 + +This notebook contains _built-in, live answer checks_! In some exercises you will see a coloured box, which runs a test case on your code, and provides feedback based on the result. Simply edit the code, run it, and the check runs again. + +_For MIT students:_ there will also be some additional (secret) test cases that will be run as part of the grading process, and we will look at your notebook and write comments. + +Feel free to ask questions! +""" + +# ╔═╡ 33e43c7c-f381-11ea-3abc-c942327456b1 +# edit the code below to set your name and kerberos ID (i.e. email without @mit.edu) + +student = (name = "Jazzy Doe", kerberos_id = "jazz") + +# you might need to wait until all other cells in this notebook have completed running. +# scroll around the page to see what's up + +# ╔═╡ ec66314e-f37f-11ea-0af4-31da0584e881 +md""" + +Submission by: **_$(student.name)_** ($(student.kerberos_id)@mit.edu) +""" + +# ╔═╡ 938185ec-f384-11ea-21dc-b56b7469f798 +md"_Let's create a package environment:_" + +# ╔═╡ 0d144802-f319-11ea-0028-cd97a776a3d0 +#img = load(download("https://upload.wikimedia.org/wikipedia/commons/thumb/a/a4/Piet_Mondriaan%2C_1930_-_Mondrian_Composition_II_in_Red%2C_Blue%2C_and_Yellow.jpg/300px-Piet_Mondriaan%2C_1930_-_Mondrian_Composition_II_in_Red%2C_Blue%2C_and_Yellow.jpg")) +#img = load(download("https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Hilma_af_Klint_-_Group_IX_SUW%2C_The_Swan_No._1_%2813947%29.jpg/477px-Hilma_af_Klint_-_Group_IX_SUW%2C_The_Swan_No._1_%2813947%29.jpg")) +img = load(download("https://i.imgur.com/4SRnmkj.png")) + +# ╔═╡ cc9fcdae-f314-11ea-1b9a-1f68b792f005 +md""" +# Arrays: Slices and views + +In the lecture (included below) we learned about what array views are. In this exercise we will add to that understanding and look at an important use of `view`s: to reduce the amount of memory allocations when reading sub-sequences within an array. + +We will use the `BenchmarkTools` package to emperically understand the effects of using views. +""" + +# ╔═╡ b49a21a6-f381-11ea-1a98-7f144c55c9b7 +html""" + +""" + +# ╔═╡ b49e8cc8-f381-11ea-1056-91668ac6ae4e +md""" +## Shrinking an array + +Below is a function called `remove_in_each_row(img, pixels)`. It takes a matrix `img` and a vector of integers, `pixels`, and shrinks the image by 1 pixel in width by removing the element `img[i, pixels[i]]` in every row. This function is one of the building blocks of the Image Seam algorithm we saw in the lecture. + +Read it and convince yourself that it is correct. +""" + +# ╔═╡ e799be82-f317-11ea-3ae4-6d13ece3fe10 +function remove_in_each_row(img, column_numbers) + @assert size(img, 1) == length(column_numbers) # same as the number of rows + m, n = size(img) + local img′ = similar(img, m, n-1) # create a similar image with one less column + + # The prime (′) in the variable name is written as \prime + # You cannot use apostrophe for this! (Apostrophe means the transpose of a matrix) + + for (i, j) in enumerate(column_numbers) + img′[i, :] = vcat(img[i, 1:j-1], img[i, j+1:end]) + end + img′ +end + +# ╔═╡ c075a8e6-f382-11ea-2263-cd9507324f4f +md"Let's use it to remove the pixels on the diagonal. These are the image dimensions before and after doing so:" + +# ╔═╡ 9cced1a8-f326-11ea-0759-0b2f22e5a1db +(before=size(img), after=size(remove_in_each_row(img, 1:size(img, 1)))) + +# ╔═╡ 1d893998-f366-11ea-0828-512de0c44915 +md""" +## **Exercise 1** - _Making it efficient_ + +We can use the `@benchmark` macro from the [BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl) package to benchmark fragments of Julia code. + +`@benchmark` takes an expression and runs it a number of times to obtain statistics about the run time and memory allocation. We generally take the minimum time as the most stable measurement of performance ([for reasons discussed in the paper on BenchmarkTools](http://www-math.mit.edu/~edelman/publications/robust_benchmarking.pdf)) +""" + +# ╔═╡ 59991872-f366-11ea-1036-afe313fb4ec1 +md""" +First, as an example, let's benchmark the `remove_in_each_row` function we defined above by passing in our image and a some indices to remove. +""" + +# ╔═╡ e501ea28-f326-11ea-252a-53949fd9ef57 +performance_experiment_default = @benchmark remove_in_each_row(img, 1:size(img, 1)) + +# ╔═╡ f7915918-f366-11ea-2c46-2f4671ae8a22 +md""" +#### Exercise 1.1 + +`vcat(x, y)` is used in julia to concatenate two arrays vertically. This actually creates a new array of size `length(x) + length(y)` and copies `x` and `y` into it. We use it in `remove_in_each_row` to create rows which have one pixel less. + +While using `vcat` might make it easy to write the first version of our function, it's strictly not necessary. + +👉 In `remove_in_each_row_no_vcat` below, figure out a way to avoid the use of `vcat` and modify the function to avoid it. +""" + +# ╔═╡ 37d4ea5c-f327-11ea-2cc5-e3774c232c2b +function remove_in_each_row_no_vcat(img, column_numbers) + @assert size(img, 1) == length(column_numbers) # same as the number of rows + m, n = size(img) + local img′ = similar(img, m, n-1) # create a similar image with one less column + + for (i, j) in enumerate(column_numbers) + # EDIT THE FOLLOWING LINE and split it into two lines + # to avoid using `vcat`. + img′[i, 1:j-1] .= img[i, 1:j-1] + img′[i, j:end] .= img[i, j+1:end] + end + img′ +end + +# ╔═╡ 67717d02-f327-11ea-0988-bfe661f57f77 +performance_experiment_without_vcat = @benchmark remove_in_each_row_no_vcat(img, 1:size(img, 1)) + +# ╔═╡ 9e149cd2-f367-11ea-28ef-b9533e8a77bb +md""" +If you did it correctly, you should see that this benchmark shows the function running faster! And "memory estimate" should also show a smaller number, and so should "allocs estimate" which is the number of allocations done per call. +""" + +# ╔═╡ ba1619d4-f389-11ea-2b3f-fd9ba71cf7e3 +md""" +#### Exercise 1.2 + +👉 How many estimated allocations did this optimization reduce, and how can you explain most of them? +""" + +# ╔═╡ e49235a4-f367-11ea-3913-f54a4a6b2d6b +no_vcat_observation = md""" + +""" + +# ╔═╡ 837c43a4-f368-11ea-00a3-990a45cb0cbd +md""" + +#### Exercise 1.3 - `view`-based optimization + +👉 In the below `remove_in_each_row_views` function, implement the same optimization to remove `vcat` and use `@view` or `@views` to avoid creating copies or slices of the `img` array. + +Pluto will automatically time your change with `@benchmark` below. +""" + +# ╔═╡ 90a22cc6-f327-11ea-1484-7fda90283797 +function remove_in_each_row_views(img, column_numbers) + @assert size(img, 1) == length(column_numbers) # same as the number of rows + m, n = size(img) + local img′ = similar(img, m, n-1) # create a similar image with one less column + + for (i, j) in enumerate(column_numbers) + # EDIT THE FOLLOWING LINE and split it into two lines + # to avoid using `vcat`. + img′[i, 1:j-1] .= @view img[i, 1:j-1] + img′[i, j:end] .= @view img[i, j+1:end] + end + img′ +end + +# ╔═╡ 3335e07c-f328-11ea-0e6c-8d38c8c0ad5b +performance_experiment_views = @benchmark begin + remove_in_each_row_views(img, 1:size(img, 1)) +end + +# ╔═╡ 40d6f562-f329-11ea-2ee4-d7806a16ede3 +md"Final tally:" + +# ╔═╡ 4f0975d8-f329-11ea-3d10-59a503f8d6b2 +( + default = performance_experiment_default, + without_vcat = performance_experiment_without_vcat, + views = performance_experiment_views, +) + +# ╔═╡ dc63d32a-f387-11ea-37e2-6f3666a72e03 +⧀(a, b) = minimum(a).allocs + size(img, 1) ÷ 2 < minimum(b).allocs; + +# ╔═╡ 7eaa57d2-f368-11ea-1a70-c7c7e54bd0b1 +md""" + +#### Exercise 1.4 + +Nice! If you did your optimizations right, you should be able to get down the estimated allocations to a single digit number! + +👉 How many allocations were avoided by adding the `@view` optimization over the `vcat` optimization? Why is this? +""" + +# ╔═╡ fd819dac-f368-11ea-33bb-17148387546a +views_observation = md""" + +""" + +# ╔═╡ 318a2256-f369-11ea-23a9-2f74c566549b +md""" +## _Brightness and Energy_ +""" + +# ╔═╡ 7a44ba52-f318-11ea-0406-4731c80c1007 +md""" +First, we will define a `brightness` function for a pixel (a color) as the mean of the red, green and blue values. + +You should use this function whenever the problem set asks you to deal with _brightness_ of a pixel. +""" + +# ╔═╡ 6c7e4b54-f318-11ea-2055-d9f9c0199341 +begin + brightness(c::RGB) = mean((c.r, c.g, c.b)) + brightness(c::RGBA) = mean((c.r, c.g, c.b)) +end + +# ╔═╡ 74059d04-f319-11ea-29b4-85f5f8f5c610 +Gray.(brightness.(img)) + +# ╔═╡ 0b9ead92-f318-11ea-3744-37150d649d43 +md"""We provide you with a convolve function below. +""" + +# ╔═╡ d184e9cc-f318-11ea-1a1e-994ab1330c1a +convolve(img, k) = imfilter(img, reflect(k)) # uses ImageFiltering.jl package +# behaves the same way as the `convolve` function used in Lecture 2 +# You were asked to implement this in homework 1. + +# ╔═╡ cdfb3508-f319-11ea-1486-c5c58a0b9177 +float_to_color(x) = RGB(max(0, -x), max(0, x), 0) + +# ╔═╡ 5fccc7cc-f369-11ea-3b9e-2f0eca7f0f0e +md""" +finally we define the `energy` function which takes the Sobel gradients along x and y directions and computes the norm of the gradient for each pixel. +""" + +# ╔═╡ 6f37b34c-f31a-11ea-2909-4f2079bf66ec +begin + energy(∇x, ∇y) = sqrt.(∇x.^2 .+ ∇y.^2) + function energy(img) + ∇y = convolve(brightness.(img), Kernel.sobel()[1]) + ∇x = convolve(brightness.(img), Kernel.sobel()[2]) + energy(∇x, ∇y) + end +end + +# ╔═╡ 9fa0cd3a-f3e1-11ea-2f7e-bd73b8e3f302 +float_to_color.(energy(img)) + +# ╔═╡ 87afabf8-f317-11ea-3cb3-29dced8e265a +md""" +## **Exercise 2** - _Building up to dynamic programming_ + +In this exercise and the following ones, we will use the computational problem of Seam carving. We will think through all the "gut reaction" solutions, and then finally end up with the dynamic programming solution that we saw in the lecture. + +In the process we will understand the performance and accuracy of each iteration of our solution. + +### How to implement the solutions: + +For every variation of the algorithm, your job is to write a function which takes a matrix of energies, and an index for a pixel on the first row, and computes a "seam" starting at that pixel. + +The function should return a vector of as many integers as there are rows in the input matrix where each number points out a pixel to delete from the corresponding row. (it acts as the input to `remove_in_each_row`). +""" + +# ╔═╡ 8ba9f5fc-f31b-11ea-00fe-79ecece09c25 +md""" +#### Exercise 2.1 - _The greedy approach_ + +The first approach discussed in the lecture (included below) is the _greedy approach_: you start from your top pixel, and at each step you just look at the three neighbors below. The next pixel in the seam is the neighbor with the lowest energy. + +""" + +# ╔═╡ f5a74dfc-f388-11ea-2577-b543d31576c6 +html""" + +""" + +# ╔═╡ c3543ea4-f393-11ea-39c8-37747f113b96 +md""" +👉 Implement the greedy approach. +""" + +# ╔═╡ 2f9cbea8-f3a1-11ea-20c6-01fd1464a592 +random_seam(m, n, i) = reduce((a, b) -> [a..., clamp(last(a) + rand(-1:1), 1, n)], 1:m-1; init=[i]) + +# ╔═╡ abf20aa0-f31b-11ea-2548-9bea4fab4c37 +function greedy_seam(energies, starting_pixel::Int) + # you can delete the body of this function - it's just a placeholder. + m, n = size(energies) + seam = Array{Int64}(undef, m) + seam[1] = starting_pixel + e = energies[1,starting_pixel] + for i in 1:m-1 + v, idx = findmin(energies[i+1,max(1,seam[i]-1):min(n,seam[i]+1)]) + e += v + seam[i+1] = seam[i] + idx - 2 + (seam[i] == 1) + end + return seam +end + +# ╔═╡ 5430d772-f397-11ea-2ed8-03ee06d02a22 +md"Before we apply your function to our test image, let's try it out on a small matrix of energies (displayed here in grayscale), just like in the lecture snippet above (clicking on the video will take you to the right part of the video). Light pixels have high energy, dark pixels signify low energy." + +# ╔═╡ f580527e-f397-11ea-055f-bb9ea8f12015 +# try +# if length(Set(greedy_seam(greedy_test, 5))) == 1 +# md"Right now you are seeing the placeholder function. (You haven't done the exercise yet!) This is a straight line from the starting pixel." +# end +# catch end + +# ╔═╡ 7ddee6fc-f394-11ea-31fc-5bd665a65bef +greedy_test = Gray.(rand(Float64, (8,10))); + +# ╔═╡ 6f52c1a2-f395-11ea-0c8a-138a77f03803 +md"Starting pixel: $(@bind greedy_starting_pixel Slider(1:size(greedy_test, 2); show_value=true))" + +# ╔═╡ 9945ae78-f395-11ea-1d78-cf6ad19606c8 +md"_Let's try it on the bigger image!_" + +# ╔═╡ 87efe4c2-f38d-11ea-39cc-bdfa11298317 +md"Compute shrunk image: $(@bind shrink_greedy CheckBox())" + +# ╔═╡ 52452d26-f36c-11ea-01a6-313114b4445d +md""" +#### Exercise 2.2 - _Recursion_ + +A common pattern in algorithm design is the idea of solving a problem as the combination of solutions to subproblems. + +The classic example, is a [Fibonacci number](https://en.wikipedia.org/wiki/Fibonacci_number) generator. + +The recursive implementation of Fibonacci looks something like this +""" + +# ╔═╡ 2a98f268-f3b6-11ea-1eea-81c28256a19e +function fib(n) + # base case (basis) + if n == 0 || n == 1 # `||` means "or" + return 1 + end + + # recursion (induction) + return fib(n-1) + fib(n-2) +end + +# ╔═╡ 32e9a944-f3b6-11ea-0e82-1dff6c2eef8d +md""" +Notice that you can call a function from within itself which may call itself and so on until a base case is reached. Then the program will combine the result from the base case up to the final result. + +In the case of the Fibonacci function, we added the solutions to the subproblems `fib(n-1)`, `fib(n-2)` to produce `fib(n)`. + +An analogy can be drawn to the process of mathematical induction in mathematics. And as with mathematical induction there are parts to constructing such a recursive algorithm: + +- Defining a base case +- Defining an recursion i.e. finding a solution to the problem as a combination of solutions to smaller problems. + +""" + +# ╔═╡ 9101d5a0-f371-11ea-1c04-f3f43b96ca4a +md""" +👉 Define a `least_energy` function which returns: +1. the lowest possible total energy for a seam starting at the pixel at $(i, j)$; +2. the column to jump to on the next move (in row $i + 1$), +which is one of $j-1$, $j$ or $j+1$, up to boundary conditions. + +Return these two values in a tuple. +""" + +# ╔═╡ 8ec27ef8-f320-11ea-2573-c97b7b908cb7 +## returns lowest possible sum energy at pixel (i, j), and the column to jump to in row i+1. +function least_energy(energies, i, j) + m, n = size(energies) + if i == m + return (energies[i,j], Int64[j]) + end + + v = energies[i,j] + les = [least_energy(energies, i+1, j2) + for j2 in max(1,j-1):min(n,j+1)] + min_v, min_idx = findmin([le[1] for le in les]) + next_idx = j + min_idx - 2 + (j == 1) + return (v + min_v, vcat([next_idx], les[min_idx][2])) +end + +# ╔═╡ a7f3d9f8-f3bb-11ea-0c1a-55bbb8408f09 +md""" +This is so elegant, correct, but inefficient! If you **check this checkbox** $(@bind compute_access CheckBox()), you will see the number of accesses made to the energies array it took to compute the least energy from the pixel (1,7): +""" + +# ╔═╡ 18e0fd8a-f3bc-11ea-0713-fbf74d5fa41a +md"Whoa!" + +# ╔═╡ cbf29020-f3ba-11ea-2cb0-b92836f3d04b +begin + struct AccessTrackerArray{T,N} <: AbstractArray{T, N} + data::Array{T,N} + accesses::Ref{Int} + end + track_access(x) = AccessTrackerArray(x, Ref(0)) + + Base.IndexStyle(::Type{AccessTrackerArray}) = IndexLinear() + + Base.size(x::AccessTrackerArray) = size(x.data) + Base.getindex(x::AccessTrackerArray, i::Int...) = (x.accesses[] += 1; x.data[i...]) + Base.setindex!(x::AccessTrackerArray, v, i...) = (x.accesses[] += 1; x.data[i...] = v;) +end + +# ╔═╡ 8bc930f0-f372-11ea-06cb-79ced2834720 +md""" +#### Exercise 2.3 - _Exhaustive search with recursion_ + +Now use the `least_energy` function you defined above to define the `recursive_seam` function which takes the energies matrix and a starting pixel, and computes the seam with the lowest energy from that starting pixel. + +This will give you the method used in the lecture to perform [exhaustive search of all possible paths](https://youtu.be/rpB6zQNsbQU?t=839). +""" + +# ╔═╡ 85033040-f372-11ea-2c31-bb3147de3c0d +function recursive_seam(energies, starting_pixel) + m, n = size(energies) + min_v, seam = least_energy(energies, 1, starting_pixel) + return seam + # Replace the following line with your code. + #seams = [least_energy(energies, 1, j) for j in 1:n] + #min_v, min_idx = findmin([seam[1] for seam in seams]) + #return seams[min_idx][2] +end + +# ╔═╡ 1d55333c-f393-11ea-229a-5b1e9cabea6a +md"Compute shrunk image: $(@bind shrink_recursive CheckBox())" + +# ╔═╡ c572f6ce-f372-11ea-3c9a-e3a21384edca +md""" +#### Exercise 2.4 + +- State clearly why this algorithm does an exhaustive search of all possible paths. +- How does the number of possible seam grow as n increases for a `m×n` image? (Big O notation is fine, or an approximation is fine). +""" + +# ╔═╡ 6d993a5c-f373-11ea-0dde-c94e3bbd1552 +exhaustive_observation = md""" + +""" + +# ╔═╡ ea417c2a-f373-11ea-3bb0-b1b5754f2fac +md""" +## **Exercise 3** - _Memoization_ + +**Memoization** is the name given to the technique of storing results to expensive function calls that will be accessed more than once. + +As stated in the video, the function `least_energy` is called repeatedly with the same arguments. In fact, we call it on the order of $3^n$ times, when there are only really $m \times n$ unique ways to call it! + +Lets implement memoization on this function with first a [dictionary](https://docs.julialang.org/en/v1/base/collections/#Dictionaries) for storage. +""" + +# ╔═╡ 56a7f954-f374-11ea-0391-f79b75195f4d +md""" +#### Exercise 3.1 - _Dictionary as storage_ + +Let's make a memoized version of least_energy function which takes a dictionary and +first checks to see if the dictionary contains the key (i,j) if it does, returns the value stored in that place, if not, will compute it, and store it in the dictionary at key (i, j) and return the value it computed. + + +`memoized_least_energy(energies, starting_pixel, memory)` + +This function must recursively call itself, and pass the same `memory` object it received as an argument. + +You are expected to read and understand the [documentation on dictionaries](https://docs.julialang.org/en/v1/base/collections/#Dictionaries) to find out how to: + +1. Create a dictionary +2. Check if a key is stored in the dictionary +3. Access contents of the dictionary by a key. +""" + +# ╔═╡ b1d09bc8-f320-11ea-26bb-0101c9a204e2 +function memoized_least_energy(energies, i, j, memory) + m, n = size(energies) + if i == m + return (energies[i,j], 0) + end + + if haskey(memory, (i, j)) + return memory[(i, j)] + end + + v = energies[i,j] + les = [memoized_least_energy(energies, i+1, j2, memory) + for j2 in max(1,j-1):min(n,j+1)] + min_v, min_idx = findmin([le[1] for le in les]) + next_idx = j + min_idx - 2 + (j == 1) + le = (v + min_v, next_idx) + memory[(i, j)] = le + return le +end + +# ╔═╡ 3e8b0868-f3bd-11ea-0c15-011bbd6ac051 +function recursive_memoized_seam(energies, starting_pixel) + memory = Dict{Tuple{Int,Int}, Tuple{Float64,Int64}}() + m, n = size(energies) + seam = Array{Int64}(undef, m) + seam[1] = starting_pixel + for i in 1:m-1 + _, seam[i+1] = memoized_least_energy(energies, i, seam[i], memory) + end + return seam +end + +# ╔═╡ 4e3bcf88-f3c5-11ea-3ada-2ff9213647b7 +md"Compute shrunk image: $(@bind shrink_dict CheckBox())" + +# ╔═╡ 820b1ec4-337e-11eb-0a09-fdbc9f2bf983 +md"Shrink by: $(@bind shrink_by Slider(1:200, show_value=true))" + +# ╔═╡ cf39fa2a-f374-11ea-0680-55817de1b837 +md""" +### Exercise 3.2 - _Matrix as storage_ + +The dictionary-based memoization we tried above works well in general as there is no restriction on what type of keys can be used. + +But in our particular case, we can use a matrix as a storage, since a matrix is naturally keyed by two integers. + +Write a variation of `matrix_memoized_least_energy` and `matrix_memoized_seam` which use a matrix as storage. +""" + +# ╔═╡ c8724b5e-f3bd-11ea-0034-b92af21ca12d +function matrix_memoized_least_energy(energies, i, j, memory) + m, n = size(energies) + if i == m + return energies[i,j] + end + + mv = memory[i,j] + if mv > -1 + return mv + end + + v = energies[i,j] + les = [matrix_memoized_least_energy(energies, i+1, j2, memory) + for j2 in max(1,j-1):min(n,j+1)] + v += minimum(les) + memory[i, j] = v + return v +end + +# ╔═╡ be7d40e2-f320-11ea-1b56-dff2a0a16e8d +function matrix_memoized_seam(energies, starting_pixel) + memory = fill(-1.0, size(energies)) + + m, n = size(energies) + seam = Array{Int64}(undef, m) + seam[1] = starting_pixel + for i in 1:m-1 + j = seam[i] + les = [matrix_memoized_least_energy(energies, i+1, j2, memory) + for j2 in max(1,j-1):min(n,j+1)] + min_v, min_idx = findmin(les) + seam[i+1] = j + min_idx - 2 + (j == 1) + end + return seam +end + +# ╔═╡ 507f3870-f3c5-11ea-11f6-ada3bb087634 +md"Compute shrunk image: $(@bind shrink_matrix CheckBox())" + +# ╔═╡ bd8974ca-3386-11eb-1bd6-956b45de09db +md"Shrink by: $(@bind matrix_shrink_by Slider(1:20, show_value=true))" + +# ╔═╡ 24792456-f37b-11ea-07b2-4f4c8caea633 +md""" +## **Exercise 4** - _Dynamic programming without recursion_ + +Now it's easy to see that the above algorithm is equivalent to one that populates the memory matrix in a for loop. + +#### Exercise 4.1 + +👉 Write a function which takes the energies and returns the least energy matrix which has the least possible seam energy for each pixel. This was shown in the lecture, but attempt to write it on your own. +""" + +# ╔═╡ ff055726-f320-11ea-32f6-2bf38d7dd310 +function least_energy_matrix(im) + m, n = size(im) + sm = Array{Float64}(undef, size(im)) + sd = Array{Int}(undef, size(im)) + sm[end, :] .= im[end, :] + for i in m-1:-1:1 + for j in 1:n + val = im[i,j] + min_below = 1000.0 + path = 0 + for k in max(1,j-1):min(n,j+1) + candidate = sm[i+1,k] + if candidate < min_below + min_below = candidate + path = k-j + end + end + sm[i,j] = min_below + val + sd[i,j] = path + end + end + return sm, sd +end + +# ╔═╡ 92e19f22-f37b-11ea-25f7-e321337e375e +md""" +#### Exercise 4.2 + +👉 Write a function which, when given the matrix returned by `least_energy_matrix` and a starting pixel (on the first row), computes the least energy seam from that pixel. +""" + +# ╔═╡ 795eb2c4-f37b-11ea-01e1-1dbac3c80c13 +function seam_from_precomputed_least_energy(energies, starting_pixel::Int) + least_energies, paths = least_energy_matrix(energies) + + m, n = size(least_energies) + seam = Array{Int64}(undef, m) + + seam[1] = starting_pixel + for i in 2:m + seam[i] = seam[i-1] + paths[i-1,seam[i-1]] + end + return seam +end + +# ╔═╡ 51df0c98-f3c5-11ea-25b8-af41dc182bac +md"Compute shrunk image: $(@bind shrink_bottomup CheckBox())" + +# ╔═╡ cbb2e2bc-344d-11eb-3bc3-a1db9e697e32 +md"Shrink by: $(@bind bottomup_n Slider(1:200, show_value=true))" + +# ╔═╡ 0fbe2af6-f381-11ea-2f41-23cd1cf930d9 +if student.kerberos_id === "jazz" + md""" +!!! danger "Oops!" + **Before you submit**, remember to fill in your name and kerberos ID at the top of this notebook! + """ +end + +# ╔═╡ 6b4d6584-f3be-11ea-131d-e5bdefcc791b +md"## Function library + +Just some helper functions used in the notebook." + +# ╔═╡ ef88c388-f388-11ea-3828-ff4db4d1874e +function mark_path(img, path) + img′ = copy(img) + m = size(img, 2) + for (i, j) in enumerate(path) + # To make it easier to see, we'll color not just + # the pixels of the seam, but also those adjacent to it + for j′ in j-1:j+1 + img′[i, clamp(j′, 1, m)] = RGB(1,0,1) + end + end + img′ +end + +# ╔═╡ 437ba6ce-f37d-11ea-1010-5f6a6e282f9b +function shrink_n(img, n, min_seam, imgs=[]; show_lightning=true) + n==0 && return push!(imgs, img) + + e = energy(img) + seam_energy(seam) = sum(e[i, seam[i]] for i in 1:size(img, 1)) + _, min_j = findmin(map(j->seam_energy(min_seam(e, j)), 1:size(e, 2))) + min_seam_vec = min_seam(e, min_j) + img′ = remove_in_each_row(img, min_seam_vec) + if show_lightning + push!(imgs, mark_path(img, min_seam_vec)) + else + push!(imgs, img′) + end + shrink_n(img′, n-1, min_seam, imgs) +end + +# ╔═╡ f6571d86-f388-11ea-0390-05592acb9195 +if shrink_greedy + greedy_carved = shrink_n(img, 200, greedy_seam) + md"Shrink by: $(@bind greedy_n Slider(1:200; show_value=true))" +end + +# ╔═╡ f626b222-f388-11ea-0d94-1736759b5f52 +if shrink_greedy + greedy_carved[greedy_n] +end + +# ╔═╡ 51e28596-f3c5-11ea-2237-2b72bbfaa001 +if shrink_bottomup + bottomup_carved = shrink_n(img, bottomup_n, seam_from_precomputed_least_energy) +end + +# ╔═╡ 0a10acd8-f3c6-11ea-3e2f-7530a0af8c7f +if shrink_bottomup + bottomup_carved[15] +end + +# ╔═╡ ef26374a-f388-11ea-0b4e-67314a9a9094 +function pencil(X) + f(x) = RGB(1-x,1-x,1-x) + map(f, X ./ maximum(X)) +end + +# ╔═╡ 6bdbcf4c-f321-11ea-0288-fb16ff1ec526 +function decimate(img, n) + img[1:n:end, 1:n:end] +end + +# ╔═╡ ddba07dc-f3b7-11ea-353e-0f67713727fc +# Do not make this image bigger, it will be infeasible to compute. +pika = decimate(load(download("https://art.pixilart.com/901d53bcda6b27b.png")),150) + +# ╔═╡ 73b52fd6-f3b9-11ea-14ed-ebfcab1ce6aa +size(pika) + +# ╔═╡ fa8e2772-f3b6-11ea-30f7-699717693164 +if compute_access + tracked = track_access(energy(pika)) + least_energy(tracked, 1,7) + tracked.accesses[] +end + +# ╔═╡ d88bc272-f392-11ea-0efd-15e0e2b2cd4e +if shrink_recursive + recursive_carved = shrink_n(pika, 3, recursive_seam) +end + +# ╔═╡ e66ef06a-f392-11ea-30ab-7160e7723a17 +if shrink_recursive + recursive_carved[recursive_n] +end + +# ╔═╡ e1273072-337f-11eb-339e-9dfcf929e8f9 +pika2 = decimate(load(download("https://art.pixilart.com/901d53bcda6b27b.png")),10)[10:end-10,10:end-10] + +# ╔═╡ ef32b196-337f-11eb-30ba-4f3f70c7af00 +size(pika2) + +# ╔═╡ 4e3ef866-f3c5-11ea-3fb0-27d1ca9a9a3f +if shrink_dict + dict_carved = shrink_n(pika2, shrink_by, recursive_memoized_seam) +end + +# ╔═╡ 6e73b1da-f3c5-11ea-145f-6383effe8a89 +if shrink_dict + dict_carved[shrink_by] +end + +# ╔═╡ 50829af6-f3c5-11ea-04a8-0535edd3b0aa +if shrink_matrix + matrix_carved = shrink_n(pika2, matrix_shrink_by, matrix_memoized_seam) +end + +# ╔═╡ 9e56ecfa-f3c5-11ea-2e90-3b1839d12038 +if shrink_matrix + matrix_carved[matrix_shrink_by] +end + +# ╔═╡ ffc17f40-f380-11ea-30ee-0fe8563c0eb1 +hint(text) = Markdown.MD(Markdown.Admonition("hint", "Hint", [text])) + +# ╔═╡ 9f18efe2-f38e-11ea-0871-6d7760d0b2f6 +hint(md"You can call the `least_energy` function recursively within itself to obtain the least energy of the adjacent cells and add the energy at the current cell to get the total energy.") + +# ╔═╡ ffc40ab2-f380-11ea-2136-63542ff0f386 +almost(text) = Markdown.MD(Markdown.Admonition("warning", "Almost there!", [text])) + +# ╔═╡ ffceaed6-f380-11ea-3c63-8132d270b83f +still_missing(text=md"Replace `missing` with your answer.") = Markdown.MD(Markdown.Admonition("warning", "Here we go!", [text])) + +# ╔═╡ ffde44ae-f380-11ea-29fb-2dfcc9cda8b4 +keep_working(text=md"The answer is not quite right.") = Markdown.MD(Markdown.Admonition("danger", "Keep working on it!", [text])) + +# ╔═╡ 980b1104-f394-11ea-0948-21002f26ee25 +function visualize_seam_algorithm(algorithm, test_img, starting_pixel) + seam = algorithm(test_img, starting_pixel) + + display_img = RGB.(test_img) + for (i, j) in enumerate(seam) + try + display_img[i, j] = RGB(0.9, 0.3, 0.6) + catch ex + if ex isa BoundsError + return keep_working("") + end + # the solution might give an illegal index + end + end + display_img +end; + +# ╔═╡ 2a7e49b8-f395-11ea-0058-013e51baa554 +visualize_seam_algorithm(greedy_seam, greedy_test, greedy_starting_pixel) + +# ╔═╡ ffe326e0-f380-11ea-3619-61dd0592d409 +yays = [md"Great!", md"Yay ❤", md"Great! 🎉", md"Well done!", md"Keep it up!", md"Good job!", md"Awesome!", md"You got the right answer!", md"Let's move on to the next section."] + +# ╔═╡ fff5aedc-f380-11ea-2a08-99c230f8fa32 +correct(text=rand(yays)) = Markdown.MD(Markdown.Admonition("correct", "Got it!", [text])) + +# ╔═╡ e3519118-f387-11ea-0c61-e1c2de1c24c1 +if performance_experiment_without_vcat ⧀ performance_experiment_default + correct() +else + keep_working(md"We are still using (roughly) the same number of allocations as the default implementation.") +end + +# ╔═╡ d4ea4222-f388-11ea-3c8d-db0d651f5282 +if performance_experiment_views ⧀ performance_experiment_default + if minimum(performance_experiment_views).allocs < 10 + correct() + else + keep_working(md"We are still using (roughly) the same number of allocations as the implementation without `vcat`.") + end +else + keep_working(md"We are still using (roughly) the same number of allocations as the default implementation.") +end + +# ╔═╡ 00026442-f381-11ea-2b41-bde1fff66011 +not_defined(variable_name) = Markdown.MD(Markdown.Admonition("danger", "Oopsie!", [md"Make sure that you define a variable called **$(Markdown.Code(string(variable_name)))**"])) + +# ╔═╡ 145c0f58-f384-11ea-2b71-09ae83f66da2 +if !@isdefined(views_observation) + not_defined(:views_observation) +end + +# ╔═╡ d7a9c000-f383-11ea-1516-cf71102d8e94 +if !@isdefined(views_observation) + not_defined(:views_observation) +end + +# ╔═╡ e0622780-f3b4-11ea-1f44-59fb9c5d2ebd +if !@isdefined(least_energy_matrix) + not_defined(:least_energy_matrix) +end + +# ╔═╡ 946b69a0-f3a2-11ea-2670-819a5dafe891 +if !@isdefined(seam_from_precomputed_least_energy) + not_defined(:seam_from_precomputed_least_energy) +end + +# ╔═╡ fbf6b0fa-f3e0-11ea-2009-573a218e2460 +function hbox(x, y, gap=16; sy=size(y), sx=size(x)) + w,h = (max(sx[1], sy[1]), + gap + sx[2] + sy[2]) + + slate = fill(RGB(1,1,1), w,h) + slate[1:size(x,1), 1:size(x,2)] .= RGB.(x) + slate[1:size(y,1), size(x,2) + gap .+ (1:size(y,2))] .= RGB.(y) + slate +end + +# ╔═╡ f010933c-f318-11ea-22c5-4d2e64cd9629 +begin + hbox( + float_to_color.(convolve(brightness.(img), Kernel.sobel()[1])), + float_to_color.(convolve(brightness.(img), Kernel.sobel()[2]))) +end + +# ╔═╡ 256edf66-f3e1-11ea-206e-4f9b4f6d3a3d +vbox(x,y, gap=16) = hbox(x', y')' + +# ╔═╡ 00115b6e-f381-11ea-0bc6-61ca119cb628 +bigbreak = html"




"; + +# ╔═╡ c086bd1e-f384-11ea-3b26-2da9e24360ca +bigbreak + +# ╔═╡ 8d558c4c-f328-11ea-0055-730ead5d5c34 +bigbreak + +# ╔═╡ f7eba2b6-f388-11ea-06ad-0b861c764d61 +bigbreak + +# ╔═╡ 4f48c8b8-f39d-11ea-25d2-1fab031a514f +bigbreak + +# ╔═╡ 48089a00-f321-11ea-1479-e74ba71df067 +bigbreak + +# ╔═╡ Cell order: +# ╟─e6b6760a-f37f-11ea-3ae1-65443ef5a81a +# ╟─ec66314e-f37f-11ea-0af4-31da0584e881 +# ╟─85cfbd10-f384-11ea-31dc-b5693630a4c5 +# ╠═33e43c7c-f381-11ea-3abc-c942327456b1 +# ╟─938185ec-f384-11ea-21dc-b56b7469f798 +# ╠═a4937996-f314-11ea-2ff9-615c888afaa8 +# ╠═0d144802-f319-11ea-0028-cd97a776a3d0 +# ╟─cc9fcdae-f314-11ea-1b9a-1f68b792f005 +# ╟─b49a21a6-f381-11ea-1a98-7f144c55c9b7 +# ╟─b49e8cc8-f381-11ea-1056-91668ac6ae4e +# ╠═e799be82-f317-11ea-3ae4-6d13ece3fe10 +# ╟─c075a8e6-f382-11ea-2263-cd9507324f4f +# ╠═9cced1a8-f326-11ea-0759-0b2f22e5a1db +# ╟─c086bd1e-f384-11ea-3b26-2da9e24360ca +# ╟─1d893998-f366-11ea-0828-512de0c44915 +# ╟─59991872-f366-11ea-1036-afe313fb4ec1 +# ╠═e501ea28-f326-11ea-252a-53949fd9ef57 +# ╟─f7915918-f366-11ea-2c46-2f4671ae8a22 +# ╠═37d4ea5c-f327-11ea-2cc5-e3774c232c2b +# ╠═67717d02-f327-11ea-0988-bfe661f57f77 +# ╟─9e149cd2-f367-11ea-28ef-b9533e8a77bb +# ╟─e3519118-f387-11ea-0c61-e1c2de1c24c1 +# ╟─ba1619d4-f389-11ea-2b3f-fd9ba71cf7e3 +# ╠═e49235a4-f367-11ea-3913-f54a4a6b2d6b +# ╟─145c0f58-f384-11ea-2b71-09ae83f66da2 +# ╟─837c43a4-f368-11ea-00a3-990a45cb0cbd +# ╠═90a22cc6-f327-11ea-1484-7fda90283797 +# ╠═3335e07c-f328-11ea-0e6c-8d38c8c0ad5b +# ╟─d4ea4222-f388-11ea-3c8d-db0d651f5282 +# ╟─40d6f562-f329-11ea-2ee4-d7806a16ede3 +# ╟─4f0975d8-f329-11ea-3d10-59a503f8d6b2 +# ╟─dc63d32a-f387-11ea-37e2-6f3666a72e03 +# ╟─7eaa57d2-f368-11ea-1a70-c7c7e54bd0b1 +# ╠═fd819dac-f368-11ea-33bb-17148387546a +# ╟─d7a9c000-f383-11ea-1516-cf71102d8e94 +# ╟─8d558c4c-f328-11ea-0055-730ead5d5c34 +# ╟─318a2256-f369-11ea-23a9-2f74c566549b +# ╟─7a44ba52-f318-11ea-0406-4731c80c1007 +# ╠═6c7e4b54-f318-11ea-2055-d9f9c0199341 +# ╠═74059d04-f319-11ea-29b4-85f5f8f5c610 +# ╟─0b9ead92-f318-11ea-3744-37150d649d43 +# ╠═d184e9cc-f318-11ea-1a1e-994ab1330c1a +# ╠═cdfb3508-f319-11ea-1486-c5c58a0b9177 +# ╟─f010933c-f318-11ea-22c5-4d2e64cd9629 +# ╟─5fccc7cc-f369-11ea-3b9e-2f0eca7f0f0e +# ╠═6f37b34c-f31a-11ea-2909-4f2079bf66ec +# ╠═9fa0cd3a-f3e1-11ea-2f7e-bd73b8e3f302 +# ╟─f7eba2b6-f388-11ea-06ad-0b861c764d61 +# ╟─87afabf8-f317-11ea-3cb3-29dced8e265a +# ╟─8ba9f5fc-f31b-11ea-00fe-79ecece09c25 +# ╟─f5a74dfc-f388-11ea-2577-b543d31576c6 +# ╟─c3543ea4-f393-11ea-39c8-37747f113b96 +# ╟─2f9cbea8-f3a1-11ea-20c6-01fd1464a592 +# ╠═abf20aa0-f31b-11ea-2548-9bea4fab4c37 +# ╟─5430d772-f397-11ea-2ed8-03ee06d02a22 +# ╟─f580527e-f397-11ea-055f-bb9ea8f12015 +# ╟─6f52c1a2-f395-11ea-0c8a-138a77f03803 +# ╠═2a7e49b8-f395-11ea-0058-013e51baa554 +# ╟─7ddee6fc-f394-11ea-31fc-5bd665a65bef +# ╟─980b1104-f394-11ea-0948-21002f26ee25 +# ╟─9945ae78-f395-11ea-1d78-cf6ad19606c8 +# ╟─87efe4c2-f38d-11ea-39cc-bdfa11298317 +# ╟─f6571d86-f388-11ea-0390-05592acb9195 +# ╟─f626b222-f388-11ea-0d94-1736759b5f52 +# ╟─52452d26-f36c-11ea-01a6-313114b4445d +# ╠═2a98f268-f3b6-11ea-1eea-81c28256a19e +# ╟─32e9a944-f3b6-11ea-0e82-1dff6c2eef8d +# ╟─9101d5a0-f371-11ea-1c04-f3f43b96ca4a +# ╠═ddba07dc-f3b7-11ea-353e-0f67713727fc +# ╠═73b52fd6-f3b9-11ea-14ed-ebfcab1ce6aa +# ╠═8ec27ef8-f320-11ea-2573-c97b7b908cb7 +# ╟─9f18efe2-f38e-11ea-0871-6d7760d0b2f6 +# ╟─a7f3d9f8-f3bb-11ea-0c1a-55bbb8408f09 +# ╟─fa8e2772-f3b6-11ea-30f7-699717693164 +# ╟─18e0fd8a-f3bc-11ea-0713-fbf74d5fa41a +# ╟─cbf29020-f3ba-11ea-2cb0-b92836f3d04b +# ╟─8bc930f0-f372-11ea-06cb-79ced2834720 +# ╠═85033040-f372-11ea-2c31-bb3147de3c0d +# ╠═1d55333c-f393-11ea-229a-5b1e9cabea6a +# ╠═e1273072-337f-11eb-339e-9dfcf929e8f9 +# ╠═ef32b196-337f-11eb-30ba-4f3f70c7af00 +# ╠═d88bc272-f392-11ea-0efd-15e0e2b2cd4e +# ╠═e66ef06a-f392-11ea-30ab-7160e7723a17 +# ╟─c572f6ce-f372-11ea-3c9a-e3a21384edca +# ╠═6d993a5c-f373-11ea-0dde-c94e3bbd1552 +# ╟─ea417c2a-f373-11ea-3bb0-b1b5754f2fac +# ╟─56a7f954-f374-11ea-0391-f79b75195f4d +# ╠═b1d09bc8-f320-11ea-26bb-0101c9a204e2 +# ╠═3e8b0868-f3bd-11ea-0c15-011bbd6ac051 +# ╟─4e3bcf88-f3c5-11ea-3ada-2ff9213647b7 +# ╠═820b1ec4-337e-11eb-0a09-fdbc9f2bf983 +# ╠═4e3ef866-f3c5-11ea-3fb0-27d1ca9a9a3f +# ╠═6e73b1da-f3c5-11ea-145f-6383effe8a89 +# ╟─cf39fa2a-f374-11ea-0680-55817de1b837 +# ╠═c8724b5e-f3bd-11ea-0034-b92af21ca12d +# ╠═be7d40e2-f320-11ea-1b56-dff2a0a16e8d +# ╟─507f3870-f3c5-11ea-11f6-ada3bb087634 +# ╠═bd8974ca-3386-11eb-1bd6-956b45de09db +# ╠═50829af6-f3c5-11ea-04a8-0535edd3b0aa +# ╠═9e56ecfa-f3c5-11ea-2e90-3b1839d12038 +# ╟─4f48c8b8-f39d-11ea-25d2-1fab031a514f +# ╟─24792456-f37b-11ea-07b2-4f4c8caea633 +# ╠═ff055726-f320-11ea-32f6-2bf38d7dd310 +# ╟─e0622780-f3b4-11ea-1f44-59fb9c5d2ebd +# ╟─92e19f22-f37b-11ea-25f7-e321337e375e +# ╠═795eb2c4-f37b-11ea-01e1-1dbac3c80c13 +# ╠═51df0c98-f3c5-11ea-25b8-af41dc182bac +# ╠═cbb2e2bc-344d-11eb-3bc3-a1db9e697e32 +# ╠═51e28596-f3c5-11ea-2237-2b72bbfaa001 +# ╠═0a10acd8-f3c6-11ea-3e2f-7530a0af8c7f +# ╟─946b69a0-f3a2-11ea-2670-819a5dafe891 +# ╟─0fbe2af6-f381-11ea-2f41-23cd1cf930d9 +# ╟─48089a00-f321-11ea-1479-e74ba71df067 +# ╟─6b4d6584-f3be-11ea-131d-e5bdefcc791b +# ╠═437ba6ce-f37d-11ea-1010-5f6a6e282f9b +# ╟─ef88c388-f388-11ea-3828-ff4db4d1874e +# ╟─ef26374a-f388-11ea-0b4e-67314a9a9094 +# ╟─6bdbcf4c-f321-11ea-0288-fb16ff1ec526 +# ╟─ffc17f40-f380-11ea-30ee-0fe8563c0eb1 +# ╟─ffc40ab2-f380-11ea-2136-63542ff0f386 +# ╟─ffceaed6-f380-11ea-3c63-8132d270b83f +# ╟─ffde44ae-f380-11ea-29fb-2dfcc9cda8b4 +# ╟─ffe326e0-f380-11ea-3619-61dd0592d409 +# ╟─fff5aedc-f380-11ea-2a08-99c230f8fa32 +# ╟─00026442-f381-11ea-2b41-bde1fff66011 +# ╟─fbf6b0fa-f3e0-11ea-2009-573a218e2460 +# ╟─256edf66-f3e1-11ea-206e-4f9b4f6d3a3d +# ╟─00115b6e-f381-11ea-0bc6-61ca119cb628 diff --git a/hw3.jl b/hw3.jl new file mode 100644 index 0000000..e8dc12b --- /dev/null +++ b/hw3.jl @@ -0,0 +1,1318 @@ +### A Pluto.jl notebook ### +# v0.12.15 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ a4937996-f314-11ea-2ff9-615c888afaa8 +begin + using Colors + using PlutoUI + using Compose + using LinearAlgebra +end + +# ╔═╡ e6b6760a-f37f-11ea-3ae1-65443ef5a81a +md"_homework 3, version 3_" + +# ╔═╡ 85cfbd10-f384-11ea-31dc-b5693630a4c5 +md""" + +# **Homework 3**: _Structure and Language_ +`18.S191`, fall 2020 + +This notebook contains _built-in, live answer checks_! In some exercises you will see a coloured box, which runs a test case on your code, and provides feedback based on the result. Simply edit the code, run it, and the check runs again. + +_For MIT students:_ there will also be some additional (secret) test cases that will be run as part of the grading process, and we will look at your notebook and write comments. + +Feel free to ask questions! +""" + +# ╔═╡ 33e43c7c-f381-11ea-3abc-c942327456b1 +# edit the code below to set your name and kerberos ID (i.e. email without @mit.edu) + +student = (name = "Jazzy Doe", kerberos_id = "jazz") + +# you might need to wait until all other cells in this notebook have completed running. +# scroll around the page to see what's up + +# ╔═╡ ec66314e-f37f-11ea-0af4-31da0584e881 +md""" + +Submission by: **_$(student.name)_** ($(student.kerberos_id)@mit.edu) +""" + +# ╔═╡ 938185ec-f384-11ea-21dc-b56b7469f798 +md"_Let's create a package environment:_" + +# ╔═╡ b49a21a6-f381-11ea-1a98-7f144c55c9b7 +html""" + +""" + +# ╔═╡ 6f9df800-f92d-11ea-2d49-c1aaabd2d012 +md""" +## **Exercise 1:** _Language detection_ + +In this exercise, we are going to create some super simple _Artificial Intelligence_. Natural language can be quite messy, but hidden in this mess is _structure_, which we are going to look for today. + +Let's start with some obvious structure in English text: the set of characters that we write the language in. If we generate random text by sampling _random Unicode characters_, it does not look like English: +""" + +# ╔═╡ b61722cc-f98f-11ea-22ae-d755f61f78c3 +String(rand(Char, 40)) + +# ╔═╡ f457ad44-f990-11ea-0e2d-2bb7627716a8 +md""" +Instead, let's define an _alphabet_, and only use those letters to sample from. To keep things simple, we ignore punctuation, capitalization, etc, and only use these 27 characters: +""" + +# ╔═╡ 4efc051e-f92e-11ea-080e-bde6b8f9295a +alphabet = ['a':'z' ; ' '] # includes the space character + +# ╔═╡ 38d1ace8-f991-11ea-0b5f-ed7bd08edde5 +md""" +Let's sample random characters from our alphabet: +""" + +# ╔═╡ ddf272c8-f990-11ea-2135-7bf1a6dca0b7 +String(rand(alphabet, 40)) |> Text + +# ╔═╡ 3cc688d2-f996-11ea-2a6f-0b4c7a5b74c2 +md""" +That already looks a lot better than our first attempt! But still, this does not look like English text -- we can do better. + +$(html"
") + +English words are not well modelled by this random-Latin-characters model. Our first observation is that **some letters are more common than others**. To put this observation into practice, we would like to have the **frequency table** of the Latin alphabet. We could search for it online, but it is actually very simple to calculate ourselves! The only thing we need is a _representative sample_ of English text. + +The following samples are from Wikipedia, but feel free to type in your own sample! You can also enter a sample of a different language, if that language can be expressed in the Latin alphabet. + +Remember that the $(html"") button on the left of a cell will show or hide the code. + +We also include a sample of Spanish, which we'll use later! +""" + +# ╔═╡ a094e2ac-f92d-11ea-141a-3566552dd839 +md""" +#### Exercise 1.1 - _Data cleaning_ +Looking at the sample, we see that it is quite _messy_: it contains punctuation, accented letters and numbers. For our analysis, we are only interested in our 27-character alphabet (i.e. `'a'` through `'z'` plus `' '`). We are going to clean the data using the Julia function `filter`. +""" + +# ╔═╡ 27c9a7f4-f996-11ea-1e46-19e3fc840ad9 +filter(isodd, [6, 7, 8, 9, -5]) + +# ╔═╡ f2a4edfa-f996-11ea-1a24-1ba78fd92233 +md""" +`filter` takes two arguments: a **function** and a **collection**. The function is applied to each element of the collection, and it returns either `true` or `false`. If the result is `true`, then that element is included in the final collection. + +Did you notice something cool? Functions are also just _objects_ in Julia, and you can use them as arguments to other functions! _(Fons thinks this is super cool.)_ + +$(html"
") + +We have written a function `isinalphabet`, which takes a character and returns a boolean: +""" + +# ╔═╡ 5c74a052-f92e-11ea-2c5b-0f1a3a14e313 +function isinalphabet(character) + character ∈ alphabet +end + +# ╔═╡ dcc4156c-f997-11ea-3e6f-057cd080d9db +isinalphabet('a'), isinalphabet('+') + +# ╔═╡ 129fbcfe-f998-11ea-1c96-0fd3ccd2dcf8 +md"👉 Use `filter` to extract just the characters from our alphabet out of `messy_sentence_1`:" + +# ╔═╡ 3a5ee698-f998-11ea-0452-19b70ed11a1d +messy_sentence_1 = "#wow 2020 ¥500 (blingbling!)" + +# ╔═╡ 75694166-f998-11ea-0428-c96e1113e2a0 +cleaned_sentence_1 = filter(isinalphabet, messy_sentence_1) + +# ╔═╡ 05f0182c-f999-11ea-0a52-3d46c65a049e +md""" +$(html"
") + +We are not interested in the case of letters (i.e. `'A'` vs `'a'`), so we want to map these to lower case _before_ we apply our filter. If we don't, all upper case letters would get deleted. +""" + +# ╔═╡ 98266882-f998-11ea-3270-4339fb502bc7 +md"👉 Use the function `lowercase` to convert `messy_sentence_2` into a lower case string, and then use `filter` to extract only the characters from our alphabet." + +# ╔═╡ d3c98450-f998-11ea-3caf-895183af926b +messy_sentence_2 = "AwesOme! 😍" + +# ╔═╡ d3a4820e-f998-11ea-2a5c-1f37e2a6dd0a +cleaned_sentence_2 = filter(isinalphabet, lowercase(messy_sentence_2)) + +# ╔═╡ aad659b8-f998-11ea-153e-3dae9514bfeb +md""" +$(html"
") + +Finally, we need to deal with **accents**: simply deleting accented characters from the source text might deform it too much. We can add accented letters to our alphabet, but a simpler solution is to *replace* accented letters with the corresponding unaccented base character. We have written a function `unaccent` that does just that. +""" + +# ╔═╡ d236b51e-f997-11ea-0c55-abb11eb35f4d +french_word = "Égalité!" + +# ╔═╡ 24860970-fc48-11ea-0009-cddee695772c +import Unicode + +# ╔═╡ 734851c6-f92d-11ea-130d-bf2a69e89255 +""" +Turn `"áéíóúüñ asdf"` into `"aeiouun asdf"`. +""" +unaccent(str) = Unicode.normalize(str, stripmark=true) + +# ╔═╡ d67034d0-f92d-11ea-31c2-f7a38ebb412f +samples = ( + English = """ +Although the word forest is commonly used, there is no universally recognised precise definition, with more than 800 definitions of forest used around the world.[4] Although a forest is usually defined by the presence of trees, under many definitions an area completely lacking trees may still be considered a forest if it grew trees in the past, will grow trees in the future,[9] or was legally designated as a forest regardless of vegetation type.[10][11] + +The word forest derives from the Old French forest (also forès), denoting "forest, vast expanse covered by trees"; forest was first introduced into English as the word denoting wild land set aside for hunting[14] without the necessity in definition of having trees on the land.[15] Possibly a borrowing, probably via Frankish or Old High German, of the Medieval Latin foresta, denoting "open wood", Carolingian scribes first used foresta in the Capitularies of Charlemagne specifically to denote the royal hunting grounds of the King. The word was not endemic to Romance languages, e. g. native words for forest in the Romance languages derived from the Latin silva, which denoted "forest" and "wood(land)" (confer the English sylva and sylvan); confer the Italian, Spanish, and Portuguese selva; the Romanian silvă; and the Old French selve, and cognates in Romance languages, e. g. the Italian foresta, Spanish and Portuguese floresta, etc., are all ultimately derivations of the French word. +""", + Spanish = """ +Un bosque es un ecosistema donde la vegetación predominante la constituyen los árboles y matas.1​ Estas comunidades de plantas cubren grandes áreas del globo terráqueo y funcionan como hábitats para los animales, moduladores de flujos hidrológicos y conservadores del suelo, constituyendo uno de los aspectos más importantes de la biosfera de la Tierra. Aunque a menudo se han considerado como consumidores de dióxido de carbono atmosférico, los bosques maduros son prácticamente neutros en cuanto al carbono, y son solamente los alterados y los jóvenes los que actúan como dichos consumidores.2​3​ De cualquier manera, los bosques maduros juegan un importante papel en el ciclo global del carbono, como reservorios estables de carbono y su eliminación conlleva un incremento de los niveles de dióxido de carbono atmosférico. + +Los bosques pueden hallarse en todas las regiones capaces de mantener el crecimiento de árboles, hasta la línea de árboles, excepto donde la frecuencia de fuego natural es demasiado alta, o donde el ambiente ha sido perjudicado por procesos naturales o por actividades humanas. Los bosques a veces contienen muchas especies de árboles dentro de una pequeña área (como la selva lluviosa tropical y el bosque templado caducifolio), o relativamente pocas especies en áreas grandes (por ejemplo, la taiga y bosques áridos montañosos de coníferas). Los bosques son a menudo hogar de muchos animales y especies de plantas, y la biomasa por área de unidad es alta comparada a otras comunidades de vegetación. La mayor parte de esta biomasa se halla en el subsuelo en los sistemas de raíces y como detritos de plantas parcialmente descompuestos. El componente leñoso de un bosque contiene lignina, cuya descomposición es relativamente lenta comparado con otros materiales orgánicos como la celulosa y otros carbohidratos. Los bosques son áreas naturales y silvestre +""" |> unaccent, +) + +# ╔═╡ a56724b6-f9a0-11ea-18f2-991e0382eccf +unaccent(french_word) + +# ╔═╡ 8d3bc9ea-f9a1-11ea-1508-8da4b7674629 +md""" +$(html"
") + +👉 Let's put everything together. Write a function `clean` that takes a string, and returns a _cleaned_ version, where: +- accented letters are replaced by their base characters; +- upper-case letters are converted to lower case; +- it is filtered to only contain characters from `alphabet` +""" + +# ╔═╡ 4affa858-f92e-11ea-3ece-258897c37e51 +function clean(text) + # we turn everything to lowercase to keep the number of letters small + return filter(isinalphabet, lowercase(unaccent(text))) +end + +# ╔═╡ e00d521a-f992-11ea-11e0-e9da8255b23b +clean("Crème brûlée est mon plat préféré.") + +# ╔═╡ 2680b506-f9a3-11ea-0849-3989de27dd9f +first_sample = clean(first(samples)) + +# ╔═╡ 571d28d6-f960-11ea-1b2e-d5977ecbbb11 +function letter_frequencies(txt) + ismissing(txt) && return missing + f = count.(string.(alphabet), txt) + f ./ sum(f) +end + +# ╔═╡ f5cbd330-3450-11eb-1d21-515e1e4f0676 +typeof(alphabet) + +# ╔═╡ f5837022-3450-11eb-118a-35ef34a042dc +string.(alphabet) + +# ╔═╡ 6a64ab12-f960-11ea-0d92-5b88943cdb1a +sample_freqs = letter_frequencies(first_sample) + +# ╔═╡ 603741c2-f9a4-11ea-37ce-1b36ecc83f45 +md""" +The result is a 27-element array, with values between `0.0` and `1.0`. These values correspond to the _frequency_ of each letter. + +`sample_freqs[i] == 0.0` means that the $i$th letter did not occur in your sample, and +`sample_freqs[i] == 0.1` means that 10% of the letters in the sample are the $i$th letter. + +To make it easier to convert between a character from the alphabet and its index, we have the following function: +""" + +# ╔═╡ b3de6260-f9a4-11ea-1bae-9153a92c3fe5 +index_of_letter(letter) = findfirst(isequal(letter), alphabet) + +# ╔═╡ a6c36bd6-f9a4-11ea-1aba-f75cecc90320 +index_of_letter('a'), index_of_letter('b'), index_of_letter(' ') + +# ╔═╡ 6d3f9dae-f9a5-11ea-3228-d147435e266d +md""" +$(html"
") + +👉 Which letters from the alphabet did not occur in the sample? +""" + +# ╔═╡ 92bf9fd2-f9a5-11ea-25c7-5966e44db6c6 +unused_letters = [alphabet[i] for i in findall(x -> x == 0.0, sample_freqs)] + +# ╔═╡ 01215e9a-f9a9-11ea-363b-67392741c8d4 +md""" +**Random letters at the correct frequencies:** +""" + +# ╔═╡ 8ae13cf0-f9a8-11ea-3919-a735c4ed9e7f +md""" +By considering the _frequencies_ of letters in English, we see that our model is already a lot better! + +Our next observation is that **some letter _combinations_ are more common than others**. Our current model thinks that `potato` is just as 'English' as `ooaptt`. In the next section, we will quantify these _transition frequencies_, and use it to improve our model. +""" + +# ╔═╡ 343d63c2-fb58-11ea-0cce-efe1afe070c2 + + +# ╔═╡ b5b8dd18-f938-11ea-157b-53b145357fd1 +function rand_sample(frequencies) + x = rand() + findfirst(z -> z >= x, cumsum(frequencies ./ sum(frequencies))) +end + +# ╔═╡ 0e872a6c-f937-11ea-125e-37958713a495 +function rand_sample_letter(frequencies) + alphabet[rand_sample(frequencies)] +end + +# ╔═╡ fbb7c04e-f92d-11ea-0b81-0be20da242c8 +function transition_counts(cleaned_sample) + [count(string(a, b), cleaned_sample) + for a in alphabet, + b in alphabet] +end + +# ╔═╡ 80118bf8-f931-11ea-34f3-b7828113ffd8 +normalize_array(x) = x ./ sum(x) + +# ╔═╡ 7f4f6ce4-f931-11ea-15a4-b3bec6a7e8b6 +transition_frequencies = normalize_array ∘ transition_counts; + +# ╔═╡ d40034f6-f9ab-11ea-3f65-7ffd1256ae9d +transition_frequencies(first_sample) + +# ╔═╡ 689ed82a-f9ae-11ea-159c-331ff6660a75 +md"What we get is a **27 by 27 matrix**. Each entry corresponds to a character pair. The _row_ corresponds to the first character, the _column_ is the second character. Let's visualize this:" + +# ╔═╡ 0b67789c-f931-11ea-113c-35e5edafcbbf +md""" +Answer the following questions with respect to the **cleaned English sample text**, which we called `first_sample`. Let's also give the transition matrix a name: +""" + +# ╔═╡ 6896fef8-f9af-11ea-0065-816a70ba9670 +sample_freq_matrix = transition_frequencies(first_sample); + +# ╔═╡ 39152104-fc49-11ea-04dd-bb34e3600f2f +if first_sample === missing + md""" + !!! danger "Don't worry!" + 👆 These errors will disappear automatically once you have completed the earlier exercises! + """ +end + +# ╔═╡ e91c6fd8-f930-11ea-01ac-476bbde79079 +md"""👉 What is the frequency of the combination `"th"`?""" + +# ╔═╡ 1b4c0c28-f9ab-11ea-03a6-69f69f7f90ed +th_frequency = sample_freq_matrix[index_of_letter('t'),index_of_letter('h')] + +# ╔═╡ 1f94e0a2-f9ab-11ea-1347-7dd906ebb09d +md"""👉 What about `"ht"`?""" + +# ╔═╡ 41b2df7c-f931-11ea-112e-ede3b16f357a +ht_frequency = sample_freq_matrix[index_of_letter('h'),index_of_letter('t')] + +# ╔═╡ 1dd1e2f4-f930-11ea-312c-5ff9e109c7f6 +md""" +👉 Which le**tt**ers appeared double in our sample? +""" + +# ╔═╡ 65c92cac-f930-11ea-20b1-6b8f45b3f262 +double_letters = filter(x -> sample_freq_matrix[index_of_letter(x),index_of_letter(x)] > 0.0, alphabet) + +# ╔═╡ 4582ebf4-f930-11ea-03b2-bf4da1a8f8df +md""" +👉 Which letter is most likely to follow a **W**? +""" + +# ╔═╡ 7898b76a-f930-11ea-2b7e-8126ec2b8ffd +most_likely_to_follow_w = let + _, idx = findmax(sample_freq_matrix[index_of_letter('w'),:]) + alphabet[idx] +end + +# ╔═╡ 458cd100-f930-11ea-24b8-41a49f6596a0 +md""" +👉 Which letter is most likely to precede a **W**? +""" + +# ╔═╡ bc401bee-f931-11ea-09cc-c5efe2f11194 +most_likely_to_precede_w = let + _, idx = findmax(sample_freq_matrix[:,index_of_letter('w')]) + alphabet[idx] +end + +# ╔═╡ 45c20988-f930-11ea-1d12-b782d2c01c11 +md""" +👉 What is the sum of each row? What is the sum of each column? How can we interpret these values?" +""" + +# ╔═╡ c4756670-3453-11eb-2ab4-712973a6d003 +sum(sample_freq_matrix, dims=1) + +# ╔═╡ f53cacf2-3453-11eb-36dd-c1b253c34cba +sum(sample_freq_matrix, dims=2) + +# ╔═╡ e69129f8-3453-11eb-0ecf-79e1a52fcd14 +sample_freqs + +# ╔═╡ cc62929e-f9af-11ea-06b9-439ac08dcb52 +row_col_answer = md""" + +""" + +# ╔═╡ 2f8dedfc-fb98-11ea-23d7-2159bdb6a299 +md""" +We can use the measured transition frequencies to generate text in a way that it has **the same transition frequencies** as our original sample. Our generated text is starting to look like real language! +""" + +# ╔═╡ b7446f34-f9b1-11ea-0f39-a3c17ba740e5 +@bind ex23_sample Select([v => String(k) for (k,v) in zip(fieldnames(typeof(samples)), samples)]) + +# ╔═╡ 4f97b572-f9b0-11ea-0a99-87af0797bf28 +md""" +**Random letters from the alphabet:** +""" + +# ╔═╡ 4e8d327e-f9b0-11ea-3f16-c178d96d07d9 +md""" +**Random letters at the correct frequencies:** +""" + +# ╔═╡ d83f8bbc-f9af-11ea-2392-c90e28e96c65 +md""" +**Random letters at the correct transition frequencies:** +""" + +# ╔═╡ 0e465160-f937-11ea-0ebb-b7e02d71e8a8 +function sample_text(A, n) + + first_index = rand_sample(vec(sum(A, dims=1))) + + indices = reduce(1:n; init=[first_index]) do word, _ + prev = last(word) + freq = normalize_array(A[prev, :]) + next = rand_sample(freq) + [word..., next] + end + + String(alphabet[indices]) +end + +# ╔═╡ 141af892-f933-11ea-1e5f-154167642809 +md""" +It looks like we have a decent language model, in the sense that it understands _transition frequencies_ in the language. In the demo above, try switching the language between $(join(string.(fieldnames(typeof(samples))), " and ")) -- the generated text clearly looks more like one or the other, demonstrating that the model can capture differences between the two languages. What's remarkable is that our "training data" was just a single paragraph per language. + +In this exercise, we will use our model to write a **classifier**: a program that automatically classifies a text as either $(join(string.(fieldnames(typeof(samples))), " or ")). + +This is not a difficult task - you can get dictionaries for both languages, and count matches - but we are doing something much more cool: we only use a single paragraph of each language, and we use a _language model_ as classifier. +""" + +# ╔═╡ 7eed9dde-f931-11ea-38b0-db6bfcc1b558 +html"

Mystery sample

+

Enter some text here -- we will detect in which language it is written!

" # dont delete me + +# ╔═╡ 7e3282e2-f931-11ea-272f-d90779264456 +@bind mystery_sample TextField((70,8), default=""" +Small boats are typically found on inland waterways such as rivers and lakes, or in protected coastal areas. However, some boats, such as the whaleboat, were intended for use in an offshore environment. In modern naval terms, a boat is a vessel small enough to be carried aboard a ship. Anomalous definitions exist, as lake freighters 1,000 feet (300 m) long on the Great Lakes are called "boats". +""") + +# ╔═╡ 7df55e6c-f931-11ea-33b8-fdc3be0b6cfa +mystery_sample + +# ╔═╡ 292e0384-fb57-11ea-0238-0fbe416fc976 +md""" +Let's compute the transition frequencies of our mystery sample! Type some text in the box below, and check whether the frequency matrix updates. +""" + +# ╔═╡ 7dabee08-f931-11ea-0cb2-c7d5afd21551 +transition_frequencies(mystery_sample) + +# ╔═╡ 3736a094-fb57-11ea-1d39-e551aae62b1d +md""" +Our model will **compare the transition frequencies of our mystery sample** to those of our two language sample. The closest match will be our detected language. + +The only question left is: How do we compare two matrices? When two matrices are almost equal, but not exactly, we want to quantify their _distance_. + +👉 Write a function called `matrix_distance` which takes 2 matrices of the same size and finds the distance between them by: + +1. Subtracting corresponding elements +2. Finding the absolute value of the difference +3. Summing the differences +""" + +# ╔═╡ 13c89272-f934-11ea-07fe-91b5d56dedf8 +function matrix_distance(A, B) + sum(abs.(A-B)) +end + +# ╔═╡ 7d60f056-f931-11ea-39ae-5fa18a955a77 +distances = map(samples) do sample + matrix_distance(transition_frequencies(mystery_sample), transition_frequencies(sample)) +end + +# ╔═╡ 7d1439e6-f931-11ea-2dab-41c66a779262 +try + @assert !ismissing(distances.English) + """

It looks like this text is **$(argmin(distances))**!

""" |> HTML +catch +end + +# ╔═╡ 8c7606f0-fb93-11ea-0c9c-45364892cbb8 +md""" +We have written a cell that selects the language with the _smallest distance_ to the mystery language. Make sure sure that `matrix_distance` is working correctly, and [scroll up](#mystery-detect) to the mystery text to see it in action! + +#### Further reading +It turns out that the SVD of the transition matrix can mysteriously group the alphabet into vowels and consonants, without any extra information. See [this paper](http://languagelog.ldc.upenn.edu/myl/Moler1983.pdf) if you want to try it yourself! We found that removing the space from `alphabet` (to match the paper) gave better results. +""" + +# ╔═╡ 82e0df62-fb54-11ea-3fff-b16c87a7d45b +md""" +## **Exercise 2** - _Language generation_ + +Our model from Exercise 1 has the property that it can easily be 'reversed' to _generate_ text. While this is useful to demonstrate its structure, the produced text is mostly meaningless: it fails to model words, let alone sentence structure. + +To take our model one step further, we are going to _generalize_ what we have done so far. Instead of looking at _letter combinations_, we will model _word combinations_. And instead of analyzing the frequencies of bigrams (combinations of two letters), we are going to analyze _$n$-grams_. + +#### Dataset +This also means that we are going to need a larger dataset to train our model on: the number of English words (and their combinations) is much higher than the number of letters. + +We will train our model on the novel [_Emma_ (1815), by Jane Austen](https://en.wikipedia.org/wiki/Emma_(novel)). This work is in the public domain, which means that we can download the whole book as a text file from `archive.org`. We've done the process of downloading and cleaning already, and we have split the text into word and punctuation tokens. +""" + +# ╔═╡ b7601048-fb57-11ea-0754-97dc4e0623a1 +emma = let + raw_text = read(download("https://ia800303.us.archive.org/24/items/EmmaJaneAusten_753/emma_pdf_djvu.txt"), String) + + first_words = "Emma Woodhouse" + last_words = "THE END" + start_index = findfirst(first_words, raw_text)[1] + stop_index = findlast(last_words, raw_text)[end] + + raw_text[start_index:stop_index] +end; + +# ╔═╡ cc42de82-fb5a-11ea-3614-25ef961729ab +function splitwords(text) + # clean up whitespace + cleantext = replace(text, r"\s+" => " ") + + # split on whitespace or other word boundaries + tokens = split(cleantext, r"(\s|\b)") +end + +# ╔═╡ d66fe2b2-fb5a-11ea-280f-cfb12b8296ac +emma_words = splitwords(emma) + +# ╔═╡ 4ca8e04a-fb75-11ea-08cc-2fdef5b31944 +forest_words = splitwords(samples.English) + +# ╔═╡ 6f613cd2-fb5b-11ea-1669-cbd355677649 +md""" +#### Exercise 2.1 - _bigrams revisited_ + +The goal of the upcoming exercises is to **generalize** what we have done in Exercise 1. To keep things simple, we _split up our problem_ into smaller problems. (The solution to any computational problem.) + +First, here is a function that takes an array, and returns the array of all **neighbour pairs** from the original. For example, + +```julia +bigrams([1, 2, 3, 42]) +``` +gives + +```julia +[[1,2], [2,3], [3,42]] +``` + +(We used integers as "words" in this example, but our function works with any type of word.) +""" + +# ╔═╡ 91e87974-fb78-11ea-3ce4-5f64e506b9d2 +function bigrams(words) + map(1:length(words)-1) do i + words[i:i+1] + end +end + +# ╔═╡ 9f98e00e-fb78-11ea-0f6c-01206e7221d6 +bigrams([1, 2, 3, 42]) + +# ╔═╡ d7d8cd0c-fb6a-11ea-12bf-2d1448b38162 +md""" +👉 Next, it's your turn to write a more general function `ngrams` that takes an array and a number $n$, and returns all **subsequences of length $n$**. For example: + +```julia +ngrams([1, 2, 3, 42], 3) +``` +should give + +```julia +[[1,2,3], [2,3,42]] +``` + +and + +```julia +ngrams([1, 2, 3, 42], 2) == bigrams([1, 2, 3, 42]) +``` +""" + +# ╔═╡ 7be98e04-fb6b-11ea-111d-51c48f39a4e9 +function ngrams(words, n) + [words[i:i+n-1] for i in 1:length(words)-n+1] +end + +# ╔═╡ 052f822c-fb7b-11ea-382f-af4d6c2b4fdb +ngrams([1, 2, 3, 42], 3) + +# ╔═╡ 067f33fc-fb7b-11ea-352e-956c8727c79f +ngrams(forest_words, 4) + +# ╔═╡ 7b10f074-fb7c-11ea-20f0-034ddff41bc3 +md""" +If you are stuck, you can write `ngrams(words, n) = bigrams(words)` (ignoring the true value of $n$), and continue with the other exercises. + +#### Exercise 2.2 - _frequency matrix revisisted_ +In Exercise 1, we use a 2D array to store the bigram frequencies, where each column or row corresponds to a character from the alphabet. If we use trigrams, we could store the frequencies in a 3D array, and so on. + +However, when counting words instead of letters, we run into a problem. A 3D array with one row, column and layer per word has too many elements to store on our computer. +""" + +# ╔═╡ 24ae5da0-fb7e-11ea-3480-8bb7b649abd5 +md""" +_Emma_ consists of $( + length(Set(emma_words)) +) unique words. This means that there are $( + Int(floor(length(Set(emma_words))^3 / 10^9)) +) billion possible trigrams - that's too much! +""" + +# ╔═╡ 47836744-fb7e-11ea-2305-3fa5819dc154 +md""" +$(html"
") + +Although the frequency array would be very large, most entries are zero. For example, _"Emma"_ is a common word, but _"Emma Emma Emma"_ does not occur in the novel. This _sparsity_ of non-zero entries can be used to **store the same information in a more efficient structure**. + +Julia's [`SparseArrays.jl` package](https://docs.julialang.org/en/v1/stdlib/SparseArrays/index.html) might sound like a logical choice, but these arrays only support 1D and 2D types, and we also want to directly index using strings, not just integers. So instead, we will use `Dict`, the dictionary type. +""" + +# ╔═╡ df4fc31c-fb81-11ea-37b3-db282b36f5ef +healthy = Dict("fruits" => ["🍎", "🍊"], "vegetables" => ["🌽", "🎃", "🍕"]) + +# ╔═╡ c83b1770-fb82-11ea-20a6-3d3a09606c62 +healthy["fruits"] + +# ╔═╡ 52970ac4-fb82-11ea-3040-8bd0590348d2 +md""" +(Did you notice something funny? The dictionary is _unordered_, this is why the entries were printed in reverse from the definition.) + +You can dynamically add or change values of a `Dict` by assigning to `my_dict[key]`. You can check whether a key already exists using `haskey(my_dict, key)`. + +👉 Use these two techniques to write a function `word_counts` that takes an array of words, and returns a `Dict` with entries `word => number_of_occurences`. + +For example: +```julia +word_counts(["to", "be", "or", "not", "to", "be"]) +``` +should return +```julia +Dict( + "to" => 2, + "be" => 2, + "or" => 1, + "not" => 1 +) +``` +""" + +# ╔═╡ 8ce3b312-fb82-11ea-200c-8d5b12f03eea +function word_counts(words::Vector) + counts = Dict() + + for word in words + if haskey(counts, word) + counts[word] += 1 + else + counts[word] = 1 + end + end + return counts +end + +# ╔═╡ a2214e50-fb83-11ea-3580-210f12d44182 +word_counts(["to", "be", "or", "not", "to", "be"]) + +# ╔═╡ 808abf6e-fb84-11ea-0785-2fc3f1c4a09f +md""" +How many times does `"Emma"` occur in the book? +""" + +# ╔═╡ 953363dc-fb84-11ea-1128-ebdfaf5160ee +emma_count = word_counts(emma_words)["Emma"] + +# ╔═╡ 294b6f50-fb84-11ea-1382-03e9ab029a2d +md""" +Great! Let's get back to our ngrams. For the purpose of generating text, we are going to store a _completions cache_. This is a dictionary where the keys are $(n-1)$-grams, and the values are all found words that complete it to an $n$-gram. Let's look at an example: + +```julia +let + trigrams = ngrams(split("to be or not to be that is the question", " "), 3) + cache = completions_cache(trigrams) + cache == Dict( + ["to", "be"] => ["or", "that"], + ["be", "or"] => ["not"], + ["or", "not"] => ["to"], + ... + ) +end +``` + +So for trigrams, our keys are the first $2$ words of each trigram, and the values are arrays containing every third word of those trigrams. + +If the same ngram occurs multiple times (e.g. "said Emma laughing"), then the last word ("laughing") should also be stored multiple times. This will allow us to generate trigrams with the same frequencies as the original text. + +👉 Write the function `completions_cache`, which takes an array of ngrams (i.e. an array of arrays of words, like the result of your `ngram` function), and returns a dictionary like described above. +""" + +# ╔═╡ b726f824-fb5e-11ea-328e-03a30544037f +function completions_cache(grams) + cache = Dict() + + for gram in grams + prefix = gram[1:length(gram)-1] + if !haskey(cache, prefix) + cache[prefix] = [] + end + push!(cache[prefix], gram[end]) + end + cache +end + +# ╔═╡ 18355314-fb86-11ea-0738-3544e2e3e816 +let + trigrams = ngrams(split("to be or not to be that is the question", " "), 3) + completions_cache(trigrams) +end + +# ╔═╡ 3d105742-fb8d-11ea-09b0-cd2e77efd15c +md""" +#### Exercise 2.4 - _write a novel_ + +We have everything we need to generate our own novel! The final step is to sample random ngrams, in a way that each next ngram overlaps with the previous one. We've done this in the function `generate_from_ngrams` below - feel free to look through the code, or to implement your own version. +""" + +# ╔═╡ a72fcf5a-fb62-11ea-1dcc-11451d23c085 +""" + generate_from_ngrams(grams, num_words) + +Given an array of ngrams (i.e. an array of arrays of words), generate a sequence of `num_words` words by sampling random ngrams. +""" +function generate_from_ngrams(grams, num_words) + n = length(first(grams)) + cache = completions_cache(grams) + + # we need to start the sequence with at least n-1 words. + # a simple way to do so is to pick a random ngram! + sequence = [rand(grams)...] + + # we iteratively add one more word at a time + for i ∈ n+1:num_words + # the previous n-1 words + tail = sequence[end-(n-2):end] + + # possible next words + completions = cache[tail] + + choice = rand(completions) + push!(sequence, choice) + end + sequence +end + +# ╔═╡ f83991c0-fb7c-11ea-0e6f-1f80709d00c1 +"Compute the ngrams of an array of words, but add the first n-1 at the end, to ensure that every ngram ends in the the beginning of another ngram." +function ngrams_circular(words, n) + ngrams([words..., words[1:n-1]...], n) +end + +# ╔═╡ abe2b862-fb69-11ea-08d9-ebd4ba3437d5 +completions_cache(ngrams_circular(forest_words, 3)) + +# ╔═╡ 4b27a89a-fb8d-11ea-010b-671eba69364e +""" + generate(source_text::AbstractString, num_token; n=3, use_words=true) + +Given a source text, generate a `String` that "looks like" the original text by satisfying the same ngram frequency distribution as the original. +""" +function generate(source_text::AbstractString, s; n=3, use_words=true) + preprocess = if use_words + splitwords + else + collect + end + + words = preprocess(source_text) + if length(words) < n + "" + else + grams = ngrams_circular(words, n) + result = generate_from_ngrams(grams, s) + if use_words + join(result, " ") + else + String(result) + end + end +end + +# ╔═╡ d7b7a14a-fb90-11ea-3e2b-2fd8f379b4d8 +md" +#### Interactive demo + +Enter your own text in the box below, and use that as training data to generate anything! +" + +# ╔═╡ 1939dbea-fb63-11ea-0bc2-2d06b2d4b26c +@bind generate_demo_sample TextField((50,5), default=samples.English) + +# ╔═╡ 70169682-fb8c-11ea-27c0-2dad2ff3080f +md"""Using $(@bind generate_sample_n_letters NumberField(1:5))grams for characters""" + +# ╔═╡ 402562b0-fb63-11ea-0769-375572cc47a8 +md"""Using $(@bind generate_sample_n_words NumberField(1:5))grams for words""" + +# ╔═╡ 2521bac8-fb8f-11ea-04a4-0b077d77529e +md""" +### Automatic Jane Austen + +Uncomment the cell below to generate some Jane Austen text: +""" + +# ╔═╡ cc07f576-fbf3-11ea-2c6f-0be63b9356fc +if student.name == "Jazzy Doe" + md""" + !!! danger "Before you submit" + Remember to fill in your **name** and **Kerberos ID** at the top of this notebook. + """ +end + +# ╔═╡ 6b4d6584-f3be-11ea-131d-e5bdefcc791b +md"## Function library + +Just some helper functions used in the notebook." + +# ╔═╡ 54b1e236-fb53-11ea-3769-b382ef8b25d6 +function Quote(text::AbstractString) + text |> Markdown.Paragraph |> Markdown.BlockQuote |> Markdown.MD +end + +# ╔═╡ b3dad856-f9a7-11ea-1552-f7435f1cb605 +String(rand(alphabet, 400)) |> Quote + +# ╔═╡ be55507c-f9a7-11ea-189c-4ffe8377212e +if sample_freqs !== missing + String([rand_sample_letter(sample_freqs) for _ in 1:400]) |> Quote +end + +# ╔═╡ 46c905d8-f9b0-11ea-36ed-0515e8ed2621 +String(rand(alphabet, 400)) |> Quote + +# ╔═╡ 489b03d4-f9b0-11ea-1de0-11d4fe4e7c69 +String([rand_sample_letter(letter_frequencies(ex23_sample)) for _ in 1:400]) |> Quote + +# ╔═╡ fd202410-f936-11ea-1ad6-b3629556b3e0 +sample_text(transition_frequencies(clean(ex23_sample)), 400) |> Quote + +# ╔═╡ b5dff8b8-fb6c-11ea-10fc-37d2a9adae8c +generate( + generate_demo_sample, 400; + n=generate_sample_n_letters, + use_words=false +) |> Quote + +# ╔═╡ ee8c5808-fb5f-11ea-19a1-3d58217f34dc +generate( + generate_demo_sample, 100; + n=generate_sample_n_words, + use_words=true +) |> Quote + +# ╔═╡ 49b69dc2-fb8f-11ea-39af-030b5c5053c3 +generate(emma, 100; n=4) |> Quote + +# ╔═╡ ddef9c94-fb96-11ea-1f17-f173a4ff4d89 +function compimg(img, labels=[c*d for c in replace(alphabet, ' ' => "_"), d in replace(alphabet, ' ' => "_")]) + xmax, ymax = size(img) + xmin, ymin = 0, 0 + arr = [(j-1, i-1) for i=1:ymax, j=1:xmax] + + compose(context(units=UnitBox(xmin, ymin, xmax, ymax)), + fill(vec(img)), + compose(context(), + fill("white"), font("monospace"), + text(first.(arr) .+ .1, last.(arr) .+ 0.6, labels)), + rectangle( + first.(arr), + last.(arr), + fill(1.0, length(arr)), + fill(1.0, length(arr)))) +end + +# ╔═╡ b7803a28-fb96-11ea-3e30-d98eb322d19a +function show_pair_frequencies(A) + imshow = let + to_rgb(x) = RGB(0.36x, 0.82x, 0.8x) + to_rgb.(A ./ maximum(abs.(A))) + end + compimg(imshow) +end + +# ╔═╡ ace3dc76-f9ae-11ea-2bee-3d0bfa57cfbc +show_pair_frequencies(transition_frequencies(first_sample)) + +# ╔═╡ ffc17f40-f380-11ea-30ee-0fe8563c0eb1 +hint(text) = Markdown.MD(Markdown.Admonition("hint", "Hint", [text])) + +# ╔═╡ 7df7ab82-f9ad-11ea-2243-21685d660d71 +hint(md"You can answer this question without writing any code: have a look at the values of `sample_freqs`.") + +# ╔═╡ e467c1c6-fbf2-11ea-0d20-f5798237c0a6 +hint(md"Start out with the same code as `bigrams`, and use the Julia documentation to learn how it works. How can we generalize the `bigram` function into the `ngram` function? It might help to do this on paper first.") + +# ╔═╡ ffc40ab2-f380-11ea-2136-63542ff0f386 +almost(text) = Markdown.MD(Markdown.Admonition("warning", "Almost there!", [text])) + +# ╔═╡ ffceaed6-f380-11ea-3c63-8132d270b83f +still_missing(text=md"Replace `missing` with your answer.") = Markdown.MD(Markdown.Admonition("warning", "Here we go!", [text])) + +# ╔═╡ ffde44ae-f380-11ea-29fb-2dfcc9cda8b4 +keep_working(text=md"The answer is not quite right.") = Markdown.MD(Markdown.Admonition("danger", "Keep working on it!", [text])) + +# ╔═╡ ffe326e0-f380-11ea-3619-61dd0592d409 +yays = [md"Fantastic!", md"Splendid!", md"Great!", md"Yay ❤", md"Great! 🎉", md"Well done!", md"Keep it up!", md"Good job!", md"Awesome!", md"You got the right answer!", md"Let's move on to the next section."] + +# ╔═╡ fff5aedc-f380-11ea-2a08-99c230f8fa32 +correct(text=rand(yays)) = Markdown.MD(Markdown.Admonition("correct", "Got it!", [text])) + +# ╔═╡ 954fc466-fb7b-11ea-2724-1f938c6b93c6 +let + output = ngrams([1, 2, 3, 42], 2) + + if output isa Missing + still_missing() + elseif !(output isa Vector{<:Vector}) + keep_working(md"Make sure that `ngrams` returns an array of arrays.") + elseif output == [[1,2], [2,3], [3,42]] + if ngrams([1,2,3], 1) == [[1],[2],[3]] + if ngrams([1,2,3], 3) == [[1,2,3]] + if ngrams(["a"],1) == [["a"]] + correct() + else + keep_working(md"`ngrams` should work with any type, not just integers!") + end + else + keep_working(md"`ngrams(x, 3)` did not give a correct result.") + end + else + keep_working(md"`ngrams(x, 1)` did not give a correct result.") + end + else + keep_working(md"`ngrams(x, 2)` did not give the correct bigrams. Start out with the same code as `bigrams`.") + end +end + +# ╔═╡ a9ffff9a-fb83-11ea-1efd-2fc15538e52f +let + output = word_counts(["to", "be", "or", "not", "to", "be"]) + + if output === nothing + keep_working(md"Did you forget to write `return`?") + elseif output == Dict() + still_missing(md"Write your function `word_counts` above.") + elseif !(output isa Dict) + keep_working(md"Make sure that `word_counts` returns a `Dict`.") + elseif output == Dict("to" => 2, "be" => 2, "or" => 1, "not" => 1) + correct() + else + keep_working() + end +end + +# ╔═╡ 00026442-f381-11ea-2b41-bde1fff66011 +not_defined(variable_name) = Markdown.MD(Markdown.Admonition("danger", "Oopsie!", [md"Make sure that you define a variable called **$(Markdown.Code(string(variable_name)))**"])) + +# ╔═╡ 6fe693c8-f9a1-11ea-1983-f159131880e9 +if !@isdefined(messy_sentence_1) + not_defined(:messy_sentence_1) +elseif !@isdefined(cleaned_sentence_1) + not_defined(:cleaned_sentence_1) +else + if cleaned_sentence_1 isa Missing + still_missing() + elseif cleaned_sentence_1 isa Vector{Char} + keep_working(md"Use `String(x)` to turn an array of characters `x` into a `String`.") + elseif cleaned_sentence_1 == filter(isinalphabet, messy_sentence_1) + correct() + else + keep_working() + end +end + +# ╔═╡ cee0f984-f9a0-11ea-2c3c-53fe26156ea4 +if !@isdefined(messy_sentence_2) + not_defined(:messy_sentence_2) +elseif !@isdefined(cleaned_sentence_2) + not_defined(:cleaned_sentence_2) +else + if cleaned_sentence_2 isa Missing + still_missing() + elseif cleaned_sentence_2 isa Vector{Char} + keep_working(md"Use `String(x)` to turn an array of characters `x` into a `String`.") + elseif cleaned_sentence_2 == filter(isinalphabet, lowercase(messy_sentence_2)) + correct() + else + keep_working() + end +end + +# ╔═╡ ddfb1e1c-f9a1-11ea-3625-f1170272e96a +if !@isdefined(clean) + not_defined(:clean) +else + let + input = "Aè !!! x1" + output = clean(input) + + + if output isa Missing + still_missing() + elseif output isa Vector{Char} + keep_working(md"Use `String(x)` to turn an array of characters `x` into a `String`.") + elseif output == "ae x" + correct() + else + keep_working() + end + end +end + +# ╔═╡ 95b81778-f9a5-11ea-3f51-019430bc8fa8 +if !@isdefined(unused_letters) + not_defined(:unused_letters) +else + if sample_freqs === missing + md""" + !!! warning "Oopsie!" + You need to complete the previous exercises first. + """ + elseif unused_letters isa Missing + still_missing() + elseif unused_letters isa String + keep_working(md"Use `collect` to turn a string into an array of characters.") + elseif Set(index_of_letter.(unused_letters)) == Set(findall(isequal(0.0), sample_freqs)) + correct() + else + keep_working() + end +end + +# ╔═╡ 489fe282-f931-11ea-3dcb-35d4f2ac8b40 +if !@isdefined(th_frequency) + not_defined(:th_frequency) +elseif !@isdefined(ht_frequency) + not_defined(:ht_frequency) +else + if th_frequency isa Missing || ht_frequency isa Missing + still_missing() + elseif th_frequency < ht_frequency + keep_working(md"Looks like your answers should be flipped. Which combination is more frequent in English?") + elseif th_frequency == sample_freq_matrix[index_of_letter('t'), index_of_letter('h')] && ht_frequency == sample_freq_matrix[index_of_letter('h'), index_of_letter('t')] + correct() + else + keep_working() + end +end + +# ╔═╡ 671525cc-f930-11ea-0e71-df9d4aae1c05 +if !@isdefined(double_letters) + not_defined(:double_letters) +end + +# ╔═╡ a5fbba46-f931-11ea-33e1-054be53d986c +if !@isdefined(most_likely_to_follow_w) + not_defined(:most_likely_to_follow_w) +end + +# ╔═╡ ba695f6a-f931-11ea-0fbb-c3ef1374270e +if !@isdefined(most_likely_to_precede_w) + not_defined(:most_likely_to_precede_w) +end + +# ╔═╡ b09f5512-fb58-11ea-2527-31bea4cee823 +if !@isdefined(matrix_distance) + not_defined(:matrix_distance) +else + try + let + A = rand(Float64, (5,4)) + B = rand(Float64, (5,4)) + + output = matrix_distance(A,B) + + if output isa Missing + still_missing() + elseif !(output isa Number) + keep_working(md"Make sure that `matrix_distance` returns a nunmber.") + elseif output == 0.0 + keep_working(md"Two different matrices should have non-zero distance.") + else + if matrix_distance(A,B) < 0 || matrix_distance(B,A) < 0 + keep_working(md"The distance between two matrices should always be positive.") + elseif matrix_distance(A,A) != 0 + almost(md"The distance between two identical matrices should be zero.") + elseif matrix_distance([1 -1], [0 0]) == 0.0 + almost(md"`matrix_distance([1 -1], [0 0])` should not be zero.") + else + correct() + end + end + end + catch + keep_working(md"The function errored.") + end +end + +# ╔═╡ 00115b6e-f381-11ea-0bc6-61ca119cb628 +bigbreak = html"




"; + +# ╔═╡ c086bd1e-f384-11ea-3b26-2da9e24360ca +bigbreak + +# ╔═╡ eaa8c79e-f9a2-11ea-323f-8bb2bd36e11c +md""" +$(bigbreak) +#### Exercise 1.2 - _Letter frequencies_ + +We are going to count the _frequency_ of each letter in this sample, after applying your `clean` function. Can you guess which character is most frequent? +""" + +# ╔═╡ dcffd7d2-f9a6-11ea-2230-b1afaecfdd54 +md""" +$(bigbreak) +Now that we know the frequencies of letters in English, we can generate random text that already looks closer to English! + +**Random letters from the alphabet:** +""" + +# ╔═╡ 77623f3e-f9a9-11ea-2f46-ff07bd27cd5f +md""" +$(bigbreak) +#### Exercise 1.3 - _Transition frequencies_ +In the previous exercise we computed the frequency of each letter in the sample by _counting_ their occurences, and then dividing by the total number of counts. + +In this exercise, we are going to count _letter transitions_, such as `aa`, `as`, `rt`, `yy`. Two letters might both be common, like `a` and `e`, but their combination, `ae`, is uncommon in English. + +To quantify this observation, we will do the same as in our last exercise: we count occurences in a _sample text_, to create the **transition frequency matrix**. +""" + +# ╔═╡ d3d7bd9c-f9af-11ea-1570-75856615eb5d +bigbreak + +# ╔═╡ 6718d26c-f9b0-11ea-1f5a-0f22f7ddffe9 +md""" +$(bigbreak) + +#### Exercise 1.4 - _Language detection_ +""" + +# ╔═╡ 568f0d3a-fb54-11ea-0f77-171718ef12a5 +bigbreak + +# ╔═╡ 7f341c4e-fb54-11ea-1919-d5421d7a2c75 +bigbreak + +# ╔═╡ Cell order: +# ╟─e6b6760a-f37f-11ea-3ae1-65443ef5a81a +# ╟─ec66314e-f37f-11ea-0af4-31da0584e881 +# ╟─85cfbd10-f384-11ea-31dc-b5693630a4c5 +# ╠═33e43c7c-f381-11ea-3abc-c942327456b1 +# ╟─938185ec-f384-11ea-21dc-b56b7469f798 +# ╠═a4937996-f314-11ea-2ff9-615c888afaa8 +# ╟─b49a21a6-f381-11ea-1a98-7f144c55c9b7 +# ╟─c086bd1e-f384-11ea-3b26-2da9e24360ca +# ╟─6f9df800-f92d-11ea-2d49-c1aaabd2d012 +# ╠═b61722cc-f98f-11ea-22ae-d755f61f78c3 +# ╟─f457ad44-f990-11ea-0e2d-2bb7627716a8 +# ╠═4efc051e-f92e-11ea-080e-bde6b8f9295a +# ╟─38d1ace8-f991-11ea-0b5f-ed7bd08edde5 +# ╠═ddf272c8-f990-11ea-2135-7bf1a6dca0b7 +# ╟─3cc688d2-f996-11ea-2a6f-0b4c7a5b74c2 +# ╟─d67034d0-f92d-11ea-31c2-f7a38ebb412f +# ╟─a094e2ac-f92d-11ea-141a-3566552dd839 +# ╠═27c9a7f4-f996-11ea-1e46-19e3fc840ad9 +# ╟─f2a4edfa-f996-11ea-1a24-1ba78fd92233 +# ╟─5c74a052-f92e-11ea-2c5b-0f1a3a14e313 +# ╠═dcc4156c-f997-11ea-3e6f-057cd080d9db +# ╟─129fbcfe-f998-11ea-1c96-0fd3ccd2dcf8 +# ╠═3a5ee698-f998-11ea-0452-19b70ed11a1d +# ╠═75694166-f998-11ea-0428-c96e1113e2a0 +# ╟─6fe693c8-f9a1-11ea-1983-f159131880e9 +# ╟─05f0182c-f999-11ea-0a52-3d46c65a049e +# ╟─98266882-f998-11ea-3270-4339fb502bc7 +# ╠═d3c98450-f998-11ea-3caf-895183af926b +# ╠═d3a4820e-f998-11ea-2a5c-1f37e2a6dd0a +# ╟─cee0f984-f9a0-11ea-2c3c-53fe26156ea4 +# ╟─aad659b8-f998-11ea-153e-3dae9514bfeb +# ╠═d236b51e-f997-11ea-0c55-abb11eb35f4d +# ╠═a56724b6-f9a0-11ea-18f2-991e0382eccf +# ╟─24860970-fc48-11ea-0009-cddee695772c +# ╟─734851c6-f92d-11ea-130d-bf2a69e89255 +# ╟─8d3bc9ea-f9a1-11ea-1508-8da4b7674629 +# ╠═4affa858-f92e-11ea-3ece-258897c37e51 +# ╠═e00d521a-f992-11ea-11e0-e9da8255b23b +# ╟─ddfb1e1c-f9a1-11ea-3625-f1170272e96a +# ╟─eaa8c79e-f9a2-11ea-323f-8bb2bd36e11c +# ╠═2680b506-f9a3-11ea-0849-3989de27dd9f +# ╠═571d28d6-f960-11ea-1b2e-d5977ecbbb11 +# ╠═f5cbd330-3450-11eb-1d21-515e1e4f0676 +# ╠═f5837022-3450-11eb-118a-35ef34a042dc +# ╠═6a64ab12-f960-11ea-0d92-5b88943cdb1a +# ╟─603741c2-f9a4-11ea-37ce-1b36ecc83f45 +# ╟─b3de6260-f9a4-11ea-1bae-9153a92c3fe5 +# ╠═a6c36bd6-f9a4-11ea-1aba-f75cecc90320 +# ╟─6d3f9dae-f9a5-11ea-3228-d147435e266d +# ╠═92bf9fd2-f9a5-11ea-25c7-5966e44db6c6 +# ╟─95b81778-f9a5-11ea-3f51-019430bc8fa8 +# ╟─7df7ab82-f9ad-11ea-2243-21685d660d71 +# ╟─dcffd7d2-f9a6-11ea-2230-b1afaecfdd54 +# ╟─b3dad856-f9a7-11ea-1552-f7435f1cb605 +# ╟─01215e9a-f9a9-11ea-363b-67392741c8d4 +# ╟─be55507c-f9a7-11ea-189c-4ffe8377212e +# ╟─8ae13cf0-f9a8-11ea-3919-a735c4ed9e7f +# ╟─343d63c2-fb58-11ea-0cce-efe1afe070c2 +# ╟─b5b8dd18-f938-11ea-157b-53b145357fd1 +# ╟─0e872a6c-f937-11ea-125e-37958713a495 +# ╟─77623f3e-f9a9-11ea-2f46-ff07bd27cd5f +# ╠═fbb7c04e-f92d-11ea-0b81-0be20da242c8 +# ╠═80118bf8-f931-11ea-34f3-b7828113ffd8 +# ╠═7f4f6ce4-f931-11ea-15a4-b3bec6a7e8b6 +# ╠═d40034f6-f9ab-11ea-3f65-7ffd1256ae9d +# ╟─689ed82a-f9ae-11ea-159c-331ff6660a75 +# ╠═ace3dc76-f9ae-11ea-2bee-3d0bfa57cfbc +# ╟─0b67789c-f931-11ea-113c-35e5edafcbbf +# ╠═6896fef8-f9af-11ea-0065-816a70ba9670 +# ╟─39152104-fc49-11ea-04dd-bb34e3600f2f +# ╟─e91c6fd8-f930-11ea-01ac-476bbde79079 +# ╠═1b4c0c28-f9ab-11ea-03a6-69f69f7f90ed +# ╟─1f94e0a2-f9ab-11ea-1347-7dd906ebb09d +# ╠═41b2df7c-f931-11ea-112e-ede3b16f357a +# ╟─489fe282-f931-11ea-3dcb-35d4f2ac8b40 +# ╟─1dd1e2f4-f930-11ea-312c-5ff9e109c7f6 +# ╠═65c92cac-f930-11ea-20b1-6b8f45b3f262 +# ╟─671525cc-f930-11ea-0e71-df9d4aae1c05 +# ╟─4582ebf4-f930-11ea-03b2-bf4da1a8f8df +# ╠═7898b76a-f930-11ea-2b7e-8126ec2b8ffd +# ╠═a5fbba46-f931-11ea-33e1-054be53d986c +# ╟─458cd100-f930-11ea-24b8-41a49f6596a0 +# ╠═bc401bee-f931-11ea-09cc-c5efe2f11194 +# ╟─ba695f6a-f931-11ea-0fbb-c3ef1374270e +# ╟─45c20988-f930-11ea-1d12-b782d2c01c11 +# ╠═c4756670-3453-11eb-2ab4-712973a6d003 +# ╠═f53cacf2-3453-11eb-36dd-c1b253c34cba +# ╠═e69129f8-3453-11eb-0ecf-79e1a52fcd14 +# ╠═cc62929e-f9af-11ea-06b9-439ac08dcb52 +# ╟─d3d7bd9c-f9af-11ea-1570-75856615eb5d +# ╟─2f8dedfc-fb98-11ea-23d7-2159bdb6a299 +# ╟─b7446f34-f9b1-11ea-0f39-a3c17ba740e5 +# ╟─4f97b572-f9b0-11ea-0a99-87af0797bf28 +# ╟─46c905d8-f9b0-11ea-36ed-0515e8ed2621 +# ╟─4e8d327e-f9b0-11ea-3f16-c178d96d07d9 +# ╟─489b03d4-f9b0-11ea-1de0-11d4fe4e7c69 +# ╟─d83f8bbc-f9af-11ea-2392-c90e28e96c65 +# ╟─fd202410-f936-11ea-1ad6-b3629556b3e0 +# ╟─0e465160-f937-11ea-0ebb-b7e02d71e8a8 +# ╟─6718d26c-f9b0-11ea-1f5a-0f22f7ddffe9 +# ╟─141af892-f933-11ea-1e5f-154167642809 +# ╟─7eed9dde-f931-11ea-38b0-db6bfcc1b558 +# ╟─7e3282e2-f931-11ea-272f-d90779264456 +# ╟─7d1439e6-f931-11ea-2dab-41c66a779262 +# ╠═7df55e6c-f931-11ea-33b8-fdc3be0b6cfa +# ╟─292e0384-fb57-11ea-0238-0fbe416fc976 +# ╠═7dabee08-f931-11ea-0cb2-c7d5afd21551 +# ╟─3736a094-fb57-11ea-1d39-e551aae62b1d +# ╠═13c89272-f934-11ea-07fe-91b5d56dedf8 +# ╟─7d60f056-f931-11ea-39ae-5fa18a955a77 +# ╟─b09f5512-fb58-11ea-2527-31bea4cee823 +# ╟─8c7606f0-fb93-11ea-0c9c-45364892cbb8 +# ╟─568f0d3a-fb54-11ea-0f77-171718ef12a5 +# ╟─82e0df62-fb54-11ea-3fff-b16c87a7d45b +# ╠═b7601048-fb57-11ea-0754-97dc4e0623a1 +# ╟─cc42de82-fb5a-11ea-3614-25ef961729ab +# ╠═d66fe2b2-fb5a-11ea-280f-cfb12b8296ac +# ╠═4ca8e04a-fb75-11ea-08cc-2fdef5b31944 +# ╟─6f613cd2-fb5b-11ea-1669-cbd355677649 +# ╠═91e87974-fb78-11ea-3ce4-5f64e506b9d2 +# ╠═9f98e00e-fb78-11ea-0f6c-01206e7221d6 +# ╟─d7d8cd0c-fb6a-11ea-12bf-2d1448b38162 +# ╠═7be98e04-fb6b-11ea-111d-51c48f39a4e9 +# ╠═052f822c-fb7b-11ea-382f-af4d6c2b4fdb +# ╠═067f33fc-fb7b-11ea-352e-956c8727c79f +# ╟─954fc466-fb7b-11ea-2724-1f938c6b93c6 +# ╟─e467c1c6-fbf2-11ea-0d20-f5798237c0a6 +# ╟─7b10f074-fb7c-11ea-20f0-034ddff41bc3 +# ╟─24ae5da0-fb7e-11ea-3480-8bb7b649abd5 +# ╟─47836744-fb7e-11ea-2305-3fa5819dc154 +# ╠═df4fc31c-fb81-11ea-37b3-db282b36f5ef +# ╠═c83b1770-fb82-11ea-20a6-3d3a09606c62 +# ╟─52970ac4-fb82-11ea-3040-8bd0590348d2 +# ╠═8ce3b312-fb82-11ea-200c-8d5b12f03eea +# ╠═a2214e50-fb83-11ea-3580-210f12d44182 +# ╟─a9ffff9a-fb83-11ea-1efd-2fc15538e52f +# ╟─808abf6e-fb84-11ea-0785-2fc3f1c4a09f +# ╠═953363dc-fb84-11ea-1128-ebdfaf5160ee +# ╟─294b6f50-fb84-11ea-1382-03e9ab029a2d +# ╠═b726f824-fb5e-11ea-328e-03a30544037f +# ╠═18355314-fb86-11ea-0738-3544e2e3e816 +# ╠═abe2b862-fb69-11ea-08d9-ebd4ba3437d5 +# ╟─3d105742-fb8d-11ea-09b0-cd2e77efd15c +# ╟─a72fcf5a-fb62-11ea-1dcc-11451d23c085 +# ╟─f83991c0-fb7c-11ea-0e6f-1f80709d00c1 +# ╟─4b27a89a-fb8d-11ea-010b-671eba69364e +# ╟─d7b7a14a-fb90-11ea-3e2b-2fd8f379b4d8 +# ╟─1939dbea-fb63-11ea-0bc2-2d06b2d4b26c +# ╟─70169682-fb8c-11ea-27c0-2dad2ff3080f +# ╟─b5dff8b8-fb6c-11ea-10fc-37d2a9adae8c +# ╟─402562b0-fb63-11ea-0769-375572cc47a8 +# ╟─ee8c5808-fb5f-11ea-19a1-3d58217f34dc +# ╟─2521bac8-fb8f-11ea-04a4-0b077d77529e +# ╠═49b69dc2-fb8f-11ea-39af-030b5c5053c3 +# ╟─7f341c4e-fb54-11ea-1919-d5421d7a2c75 +# ╟─cc07f576-fbf3-11ea-2c6f-0be63b9356fc +# ╟─6b4d6584-f3be-11ea-131d-e5bdefcc791b +# ╟─54b1e236-fb53-11ea-3769-b382ef8b25d6 +# ╟─b7803a28-fb96-11ea-3e30-d98eb322d19a +# ╟─ddef9c94-fb96-11ea-1f17-f173a4ff4d89 +# ╟─ffc17f40-f380-11ea-30ee-0fe8563c0eb1 +# ╟─ffc40ab2-f380-11ea-2136-63542ff0f386 +# ╟─ffceaed6-f380-11ea-3c63-8132d270b83f +# ╟─ffde44ae-f380-11ea-29fb-2dfcc9cda8b4 +# ╟─ffe326e0-f380-11ea-3619-61dd0592d409 +# ╟─fff5aedc-f380-11ea-2a08-99c230f8fa32 +# ╟─00026442-f381-11ea-2b41-bde1fff66011 +# ╟─00115b6e-f381-11ea-0bc6-61ca119cb628 diff --git a/seam_carving.jl b/seam_carving.jl new file mode 100644 index 0000000..15ee27f --- /dev/null +++ b/seam_carving.jl @@ -0,0 +1,553 @@ +### A Pluto.jl notebook ### +# v0.12.15 + +using Markdown +using InteractiveUtils + +# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error). +macro bind(def, element) + quote + local el = $(esc(element)) + global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing + el + end +end + +# ╔═╡ 877df834-f078-11ea-303b-e98273ef98a4 +begin + using Pkg + Pkg.activate(tempname()) +end + +# ╔═╡ 0316b94c-eef6-11ea-19bc-dbc959901bb5 +begin + using Images + using ImageMagick + using Statistics + using LinearAlgebra + using ImageFiltering +end + +# ╔═╡ fe19ad0a-ef04-11ea-1e5f-1bfcbbb51302 +using PlutoUI + +# ╔═╡ e196fa66-eef5-11ea-2afe-c9fcb6c48937 +# Poor man's Project.toml + +Pkg.add(["Images", + "ImageMagick", + "PlutoUI", + "Hyperscript", + "ImageFiltering"]) + + +# ╔═╡ cb335074-eef7-11ea-24e8-c39a325166a1 +md""" +# Seam Carving + +1. We use convolution with Sobel filters for "edge detection". +2. We use that to write an algorithm that removes "uninteresting" + bits of an image in order to shrink it. +""" + +# ╔═╡ d2ae6dd2-eef9-11ea-02df-255ec3b46a36 +begin + example_urls = [ +"https://cdn.shortpixel.ai/spai/w_1086+q_lossy+ret_img+to_webp/https://wisetoast.com/wp-content/uploads/2015/10/The-Persistence-of-Memory-salvador-deli-painting.jpg", + +"https://upload.wikimedia.org/wikipedia/commons/thumb/1/17/Gustave_Caillebotte_-_Paris_Street%3B_Rainy_Day_-_Google_Art_Project.jpg/1014px-Gustave_Caillebotte_-_Paris_Street%3B_Rainy_Day_-_Google_Art_Project.jpg", + +"https://upload.wikimedia.org/wikipedia/commons/thumb/1/17/Gustave_Caillebotte_-_Paris_Street%3B_Rainy_Day_-_Google_Art_Project.jpg/1014px-Gustave_Caillebotte_-_Paris_Street%3B_Rainy_Day_-_Google_Art_Project.jpg", + +"https://upload.wikimedia.org/wikipedia/commons/thumb/c/cc/Grant_Wood_-_American_Gothic_-_Google_Art_Project.jpg/480px-Grant_Wood_-_American_Gothic_-_Google_Art_Project.jpg", + "https://cdn.shortpixel.ai/spai/w_1086+q_lossy+ret_img+to_webp/https://wisetoast.com/wp-content/uploads/2015/10/The-Persistence-of-Memory-salvador-deli-painting.jpg", + +"https://upload.wikimedia.org/wikipedia/commons/thumb/7/7d/A_Sunday_on_La_Grande_Jatte%2C_Georges_Seurat%2C_1884.jpg/640px-A_Sunday_on_La_Grande_Jatte%2C_Georges_Seurat%2C_1884.jpg", + +"https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg/758px-Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg", + "https://web.mit.edu/facilities/photos/construction/Projects/stata/1_large.jpg", + ] + img = load(download(example_urls[2])) +end + +# ╔═╡ 8340cf20-f079-11ea-1665-f5864bc49cb9 + + +# ╔═╡ 0b6010a8-eef6-11ea-3ad6-c1f10e30a413 +# arbitrarily choose the brightness of a pixel as mean of rgb +# brightness(c::AbstractRGB) = mean((c.r, c.g, c.b)) + +# Use a weighted sum of rgb giving more weight to colors we perceive as 'brighter' +# Based on https://www.tutorialspoint.com/dip/grayscale_to_rgb_conversion.htm +brightness(c::AbstractRGB) = 0.3 * c.r + 0.59 * c.g + 0.11 * c.b + +# ╔═╡ fc1c43cc-eef6-11ea-0fc4-a90ac4336964 +Gray.(brightness.(img)) + +# ╔═╡ 82c0d0c8-efec-11ea-1bb9-83134ecb877e +md""" +# Edge detection filter + +(Spoiler alert!) Here, we use the Sobel edge detection filter we created in Homework 1. + +```math +\begin{align} + +G_x &= \begin{bmatrix} +1 & 0 & -1 \\ +2 & 0 & -2 \\ +1 & 0 & -1 \\ +\end{bmatrix}*A\\ +G_y &= \begin{bmatrix} +1 & 2 & 1 \\ +0 & 0 & 0 \\ +-1 & -2 & -1 \\ +\end{bmatrix}*A +\end{align} +``` +Here $A$ is the array corresponding to your image. +We can think of these as derivatives in the $x$ and $y$ directions. + +Then we combine them by finding the magnitude of the **gradient** (in the sense of multivariate calculus) by defining + +$$G_\text{total} = \sqrt{G_x^2 + G_y^2}.$$ +""" + +# ╔═╡ da726954-eff0-11ea-21d4-a7f4ae4a6b09 +Sy, Sx = Kernel.sobel() + +# ╔═╡ abf6944e-f066-11ea-18e2-0b92606dab85 +(collect(Int.(8 .* Sy)), collect(Int.(8 .* Sx))) + +# ╔═╡ ac8d6902-f069-11ea-0f1d-9b0fa706d769 +md""" +- blue shows positive values +- red shows negative values + $G_x \hspace{180pt} G_y$ +""" + +# ╔═╡ 172c7612-efee-11ea-077a-5d5c6e2505a4 +function shrink_image(image, ratio=5) + (height, width) = size(image) + new_height = height ÷ ratio - 1 + new_width = width ÷ ratio - 1 + list = [ + mean(image[ + ratio * i:ratio * (i + 1), + ratio * j:ratio * (j + 1), + ]) + for j in 1:new_width + for i in 1:new_height + ] + reshape(list, new_height, new_width) +end + +# ╔═╡ fcf46120-efec-11ea-06b9-45f470899cb2 +function convolve(M, kernel) + height, width = size(kernel) + + half_height = height ÷ 2 + half_width = width ÷ 2 + + new_image = similar(M) + + # (i, j) loop over the original image + m, n = size(M) + @inbounds for i in 1:m + for j in 1:n + # (k, l) loop over the neighbouring pixels + accumulator = 0 * M[1, 1] + for k in -half_height:-half_height + height - 1 + for l in -half_width:-half_width + width - 1 + Mi = i - k + Mj = j - l + # First index into M + if Mi < 1 + Mi = 1 + elseif Mi > m + Mi = m + end + # Second index into M + if Mj < 1 + Mj = 1 + elseif Mj > n + Mj = n + end + + accumulator += kernel[k, l] * M[Mi, Mj] + end + end + new_image[i, j] = accumulator + end + end + + return new_image +end + +# ╔═╡ 6f7bd064-eff4-11ea-0260-f71aa7f4f0e5 +function edgeness(img) + Sy, Sx = Kernel.sobel() + b = brightness.(img) + + ∇y = convolve(b, Sy) + ∇x = convolve(b, Sx) + + sqrt.(∇x.^2 + ∇y.^2) +end + +# ╔═╡ dec62538-efee-11ea-1e03-0b801e61e91c + function show_colored_array(array) + pos_color = RGB(0.36, 0.82, 0.8) + neg_color = RGB(0.99, 0.18, 0.13) + to_rgb(x) = max(x, 0) * pos_color + max(-x, 0) * neg_color + to_rgb.(array) / maximum(abs.(array)) + end + +# ╔═╡ da39c824-eff0-11ea-375b-1b6c6e186182 +# Sx +# collect(Int.(8 .* Sx)) +show_colored_array(Sx) + +# ╔═╡ 074a58be-f146-11ea-382c-b7ae6c44bf75 +# Sy +# collect(Int.(8 .* Sy)) +show_colored_array(Sy) + +# ╔═╡ f8283a0e-eff4-11ea-23d3-9f1ced1bafb4 +md""" + +## Seam carving idea + +The idea of seam carving is to find a path from the top of the image to the bottom of the image where the path minimizes the edgness. + +In other words, this path **minimizes the number of edges it crosses** +""" + +# ╔═╡ 025e2c94-eefb-11ea-12cb-f56f34886334 +md""" + +At every step in going down, the path is allowed to go south west, south or south east. We want to find a seam with the minimum possible sum of energies. + +We start by writing a `least_edgy` function which given a matrix of energies, returns +a matrix of minimum possible energy starting from that pixel going up to a pixel in the bottom most row. +""" + +# ╔═╡ e5a7e426-2d0c-11eb-07fd-cd587148bd38 +small_img = shrink_image(img, 10) + +# ╔═╡ eff57fca-2d06-11eb-0ef2-f94561e916eb +function get_seam_map(im) + m, n = size(im) + sm = Array{Float64}(undef, size(im)) + sd = Array{Int}(undef, size(im)) + sm[end, :] .= im[end, :] + for i in m-1:-1:1 + for j in 1:n + val = im[i,j] + min_below = 1000.0 + path = 0 + for k in max(1,j-1):min(n,j+1) + candidate = sm[i+1,k] + if candidate < min_below + min_below = candidate + path = k-j + end + end + sm[i,j] = min_below + val + sd[i,j] = path + end + end + return sm, sd +end + +# ╔═╡ 1c01264c-2d08-11eb-0549-076c31bb8fa6 +sm, sd = get_seam_map(edgeness(img)) + +# ╔═╡ acc1ee8c-eef9-11ea-01ac-9b9e9c4167b3 +# e[x,y] +# ↙ ↓ ↘ <--pick the next path which gives the least overall energy +# e[x-1,y+1] e[x,y+1] e[x+1,y+1] +# +# Basic Comp: e[x,y] += min( e[x-1,y+1],e[x,y],e[x+1,y]) +# dirs records which one from (-1==SW,0==S,1==SE) + +function least_edgy(E) + least_E = zeros(size(E)) + dirs = zeros(Int, size(E)) + least_E[end, :] .= E[end, :] # the minimum energy on the last row is the energy + # itself + + m, n = size(E) + # Go from the last row up, finding the minimum energy + for i in m-1:-1:1 + for j in 1:n + j1, j2 = max(1, j-1), min(j+1, n) + e, dir = findmin(least_E[i+1, j1:j2]) + least_E[i,j] += e + least_E[i,j] += E[i,j] + dirs[i, j] = (-1,0,1)[dir + (j==1)] + end + end + least_E, dirs +end + +# ╔═╡ 8b204a2a-eff6-11ea-25b0-13f230037ee1 +# The bright areas are screaming "AVOID ME!!!" +least_e, dirs = least_edgy(edgeness(img)) + +# ╔═╡ 84d3afe4-eefe-11ea-1e31-bf3b2af4aecd +show_colored_array(least_e) + +# ╔═╡ 50a49b56-2d09-11eb-2d33-8d4cd5896e38 +show_colored_array(sm) + +# ╔═╡ b507480a-ef01-11ea-21c4-63d19fac19ab +# direction the path should take at every pixel. +reduce((x,y)->x*y*"\n", + reduce(*, getindex.(([" ", "↙", "↓", "↘"],), dirs[1:25, 1:76].+3), dims=2, init=""), init="") |> Text + +# ╔═╡ 696cf94e-2d09-11eb-136a-8f360d3a7507 +reduce((x,y)->x*y*"\n", + reduce(*, getindex.(([" ", "↙", "↓", "↘"],), sd[1:25, 1:76].+3), dims=2, init=""), init="") |> Text + +# ╔═╡ 7d8b20a2-ef03-11ea-1c9e-fdf49a397619 +md"## Remove seams" + +# ╔═╡ f690b06a-ef31-11ea-003b-4f2b2f82a9c3 +md""" +Compressing an image horizontally involves a number of seams of lowest energy successively. +""" + +# ╔═╡ 0adf2e18-2d10-11eb-2a27-6de83b5aad3a +function get_seam_at2(sd, j) + h = size(sd, 1) + js = Array{Int}(undef, h) + js[1] = j + for i=2:h + js[i] = js[i-1] + sd[i-1,js[i-1]] + end + tuple.(1:h, js) +end + +# ╔═╡ 977b6b98-ef03-11ea-0176-551fc29729ab +function get_seam_at(dirs, j) + m = size(dirs, 1) + js = fill(0, m) + js[1] = j + for i=2:m + js[i] = js[i-1] + dirs[i-1, js[i-1]] + end + tuple.(1:m, js) +end + +# ╔═╡ 9abbb158-ef03-11ea-39df-a3e8aa792c50 +get_seam_at(dirs, 2) + +# ╔═╡ fe709508-2d10-11eb-02a9-29080b78d44a +get_seam_at2(dirs, 2) + +# ╔═╡ 14f72976-ef05-11ea-2ad5-9f0914f9cf58 +function mark_path(img, path) + img′ = copy(img) + m = size(img, 2) + for (i, j) in path + # To make it easier to see, we'll color not just + # the pixels of the seam, but also those adjacent to it + for j′ in j-1:j+1 + img′[i, clamp(j′, 1, m)] = RGB(1,0,1) + end + end + img′ +end + +# ╔═╡ cf9a9124-ef04-11ea-14a4-abf930edc7cc +@bind start_column Slider(1:size(img, 2)) + +# ╔═╡ 772a4d68-ef04-11ea-366a-f7ae9e1634f6 +path = get_seam_at2(dirs, start_column) + +# ╔═╡ 081a98cc-f06e-11ea-3664-7ba51d4fd153 +function pencil(X) + f(x) = RGB(1-x,1-x,1-x) + map(f, X ./ maximum(X)) +end + +# ╔═╡ 237647e8-f06d-11ea-3c7e-2da57e08bebc +e = edgeness(img); + +# ╔═╡ c7a7386e-2d11-11eb-3885-056f6d7fb840 + + +# ╔═╡ 4f23bc54-ef0f-11ea-06a9-35ca3ece421e +function rm_path(img, path) + img′ = img[:, 1:end-1] # one less column + for (i, j) in path + img′[i, 1:j-1] .= img[i, 1:j-1] + img′[i, j:end] .= img[i, j+1:end] + end + img′ +end + +# ╔═╡ b401f398-ef0f-11ea-38fe-012b7bc8a4fa +function shrink_n(img, n) + imgs = [] + marked_imgs = [] + + e = edgeness(img) + for i=1:n + least_E, dirs = get_seam_map(e) + _, min_j = findmin(@view least_E[1, :]) + seam = get_seam_at2(dirs, min_j) + img = rm_path(img, seam) + # Recompute the energy for the new image + # Note, this currently involves rerunning the convolution + # on the whole image, but in principle the only values that + # need recomputation are those adjacent to the seam, so there + # is room for a meanintful speedup here. + e = edgeness(img) + e = rm_path(e, seam) + + push!(imgs, img) + push!(marked_imgs, mark_path(img, seam)) + end + imgs, marked_imgs +end + +# ╔═╡ b1b6b7fc-f153-11ea-224a-2578e8298775 +n_examples = min(200, size(img, 2)) + +# ╔═╡ 2eb459d4-ef36-11ea-1f74-b53ffec7a1ed +# returns two vectors of n successively smaller images +# The second images have markings where the seam is cut out +carved, marked_carved = shrink_n(img, n_examples); + +# ╔═╡ 7038abe4-ef36-11ea-11a5-75e57ab51032 +@bind n Slider(1:length(carved)) + +# ╔═╡ 2d6c6820-ef2d-11ea-1704-49bb5188cfcc +md"shrunk by $n:" + +# ╔═╡ 1fd26a60-f089-11ea-1f56-bb6eba7d9651 +function hbox(x, y, gap=16; sy=size(y), sx=size(x)) + w,h = (max(sx[1], sy[1]), + gap + sx[2] + sy[2]) + + slate = fill(RGB(1,1,1), w,h) + slate[1:size(x,1), 1:size(x,2)] .= RGB.(x) + slate[1:size(y,1), size(x,2) + gap .+ (1:size(y,2))] .= RGB.(y) + slate +end + +# ╔═╡ 44192a40-eff2-11ea-0ec7-05cdadb0c29a +begin + img_brightness = brightness.(img) + ∇x = convolve(img_brightness, Sx) + ∇y = convolve(img_brightness, Sy) + hbox(show_colored_array(∇x), show_colored_array(∇y)) +end + +# ╔═╡ d6a268c0-eff4-11ea-2c9e-bfef19c7f540 +begin + edged = edgeness(img) + # hbox(img, pencil(edged)) + hbox(img, Gray.(edgeness(img)) / maximum(abs.(edged))) +end + +# ╔═╡ 552fb92e-ef05-11ea-0a79-dd7a6760089a +hbox(mark_path(img, path), mark_path(show_colored_array(least_e), path)) + +# ╔═╡ dfd03c4e-f06c-11ea-1e2a-89233a675138 +let + hbox(mark_path(img, path), mark_path(pencil(e), path)); +end + +# ╔═╡ ca4a87e8-eff8-11ea-3d57-01dfa34ff723 +let + # least energy path of them all: + _, k = findmin(least_e[1, :]) + path = get_seam_at(dirs, k) + hbox( + mark_path(img, path), + mark_path(show_colored_array(least_e), path) + ) +end + +# ╔═╡ fa6a2152-ef0f-11ea-0e67-0d1a6599e779 +hbox(img, marked_carved[n], sy=size(img)) + +# ╔═╡ 71b16dbe-f08b-11ea-2343-5f1583074029 +vbox(x,y, gap=16) = hbox(x', y')' + +# ╔═╡ ddac52ea-f148-11ea-2860-21cff4c867e6 +let + ∇y = convolve(brightness.(img), Sy) + ∇x = convolve(brightness.(img), Sx) + # zoom in on the clock + vbox( + hbox(img[300:end, 1:300], img[300:end, 1:300]), + hbox(show_colored_array.((∇x[300:end, 1:300], ∇y[300:end, 1:300]))...) + ) +end + +# ╔═╡ 15d1e5dc-ef2f-11ea-093a-417108bcd495 +[size(img) size(carved[n])] + +# ╔═╡ Cell order: +# ╠═877df834-f078-11ea-303b-e98273ef98a4 +# ╠═e196fa66-eef5-11ea-2afe-c9fcb6c48937 +# ╠═0316b94c-eef6-11ea-19bc-dbc959901bb5 +# ╟─cb335074-eef7-11ea-24e8-c39a325166a1 +# ╠═d2ae6dd2-eef9-11ea-02df-255ec3b46a36 +# ╠═8340cf20-f079-11ea-1665-f5864bc49cb9 +# ╠═0b6010a8-eef6-11ea-3ad6-c1f10e30a413 +# ╠═fc1c43cc-eef6-11ea-0fc4-a90ac4336964 +# ╟─82c0d0c8-efec-11ea-1bb9-83134ecb877e +# ╠═da726954-eff0-11ea-21d4-a7f4ae4a6b09 +# ╠═da39c824-eff0-11ea-375b-1b6c6e186182 +# ╠═074a58be-f146-11ea-382c-b7ae6c44bf75 +# ╠═abf6944e-f066-11ea-18e2-0b92606dab85 +# ╠═44192a40-eff2-11ea-0ec7-05cdadb0c29a +# ╟─ac8d6902-f069-11ea-0f1d-9b0fa706d769 +# ╠═ddac52ea-f148-11ea-2860-21cff4c867e6 +# ╠═6f7bd064-eff4-11ea-0260-f71aa7f4f0e5 +# ╟─d6a268c0-eff4-11ea-2c9e-bfef19c7f540 +# ╠═172c7612-efee-11ea-077a-5d5c6e2505a4 +# ╟─fcf46120-efec-11ea-06b9-45f470899cb2 +# ╟─dec62538-efee-11ea-1e03-0b801e61e91c +# ╟─f8283a0e-eff4-11ea-23d3-9f1ced1bafb4 +# ╟─025e2c94-eefb-11ea-12cb-f56f34886334 +# ╠═e5a7e426-2d0c-11eb-07fd-cd587148bd38 +# ╠═eff57fca-2d06-11eb-0ef2-f94561e916eb +# ╠═1c01264c-2d08-11eb-0549-076c31bb8fa6 +# ╠═acc1ee8c-eef9-11ea-01ac-9b9e9c4167b3 +# ╠═8b204a2a-eff6-11ea-25b0-13f230037ee1 +# ╠═84d3afe4-eefe-11ea-1e31-bf3b2af4aecd +# ╠═50a49b56-2d09-11eb-2d33-8d4cd5896e38 +# ╠═b507480a-ef01-11ea-21c4-63d19fac19ab +# ╠═696cf94e-2d09-11eb-136a-8f360d3a7507 +# ╟─7d8b20a2-ef03-11ea-1c9e-fdf49a397619 +# ╠═f690b06a-ef31-11ea-003b-4f2b2f82a9c3 +# ╠═fe19ad0a-ef04-11ea-1e5f-1bfcbbb51302 +# ╠═0adf2e18-2d10-11eb-2a27-6de83b5aad3a +# ╠═977b6b98-ef03-11ea-0176-551fc29729ab +# ╠═9abbb158-ef03-11ea-39df-a3e8aa792c50 +# ╠═fe709508-2d10-11eb-02a9-29080b78d44a +# ╠═772a4d68-ef04-11ea-366a-f7ae9e1634f6 +# ╟─14f72976-ef05-11ea-2ad5-9f0914f9cf58 +# ╠═cf9a9124-ef04-11ea-14a4-abf930edc7cc +# ╠═552fb92e-ef05-11ea-0a79-dd7a6760089a +# ╠═081a98cc-f06e-11ea-3664-7ba51d4fd153 +# ╠═237647e8-f06d-11ea-3c7e-2da57e08bebc +# ╠═dfd03c4e-f06c-11ea-1e2a-89233a675138 +# ╠═ca4a87e8-eff8-11ea-3d57-01dfa34ff723 +# ╠═c7a7386e-2d11-11eb-3885-056f6d7fb840 +# ╠═4f23bc54-ef0f-11ea-06a9-35ca3ece421e +# ╠═b401f398-ef0f-11ea-38fe-012b7bc8a4fa +# ╠═b1b6b7fc-f153-11ea-224a-2578e8298775 +# ╠═2eb459d4-ef36-11ea-1f74-b53ffec7a1ed +# ╠═7038abe4-ef36-11ea-11a5-75e57ab51032 +# ╟─2d6c6820-ef2d-11ea-1704-49bb5188cfcc +# ╠═fa6a2152-ef0f-11ea-0e67-0d1a6599e779 +# ╟─71b16dbe-f08b-11ea-2343-5f1583074029 +# ╟─1fd26a60-f089-11ea-1f56-bb6eba7d9651 +# ╟─15d1e5dc-ef2f-11ea-093a-417108bcd495