Tuesday 23 January 2024

Automatic Speech Recognition (Part II)

WFST

decoding/recognition

 

A lattice is a representation of the alternative word-sequences that are "sufficiently likely" for a particular utterance. In order to understand lattices properly you have to understand decoding graphs in the WFST framework

 

SPEECH RECOGNITION WITH WEIGHTED FINITE-STATE TRANSDUCERS

https://cs.nyu.edu/~mohri/pub/hbka.pdf

Audio Intro

https://ketanhdoshi.github.io/Audio-Intro/

 

sound:  variation in air pressure plotted over time

 

  • Amplitude is loudness (or "volume".) High amplitude is loud, low amplitude is quiet. We measure loudness in decibels (db).
  • Frequency is pitch. High frequency is a high-pitched sound, low frequency is, well, low. We measure frequency in hertz (Hz) and kilohertz (kHz), which is thousands of hertz.

 

Fourier Transform: convert audio from Time-domain (Amplitude vs. Time) to  Frequency-domain (Frequency vs. Time)

 

The Spectrum plots all of the frequencies that are present in the signal along with the strength or amplitude of each frequency.

It is useful to convert audio waveform into an image such as spectrogram so that CNN-architecture to extract features.

 

 

Thursday 5 October 2023

The Monocle Companion: Fifty ideas for a BETTER WORLD

Out of the 50 ideas discussed in the book, please find below a dozen ideas that struck a chord with me: 

1. Have a holiday romance by Annabel Streets

2. The nothing paradox by Eliane Glaser: The Art of Doing Nothing

3. How not to be an expat by Pallavi Aiyar

4. Ode to a lie-in by Fernando Augusto Pacheco

5. In praise of silence by Mary Fitzgerald

6. Obtaining 'attunement' by Christopher Lord

7. The power of recommendation by Saul Taylor

8. Taking the plunge by Francheska Melendez

9. A walk in the woods by Ariel Sophia Bardi

10. The blame game by Sally O'Sullivan

11. How to make friends by David Sax

12. In defence of commute by Lilian Fawcett

Sunday 23 July 2023

On `Death by PowerPoint'

Preparing the Slides
  • think about your slides last
  • create a consistent look and feel
  • avoid slides with lots of text
  • use simple photos that enhance meaning
  • use storytelling
  • have a focused message that you want your audience to retain; 1 slide = 1 message
  • know your audience
Presenting the Slides
  • glance technology: take a pause between slides; let the audience take a look at the slides and then listen to the speaker for details
  • create a connection with the audience
Corporate vs. Academic Presentation
  • learn from TED talks how to present to a wide audience
  • "I noticed that in academia we tend to emphasize methodology for about 10 slides, and in the industry, they only have one slide of the method and 10 slides of results and implication of results aka. discussion."
  • leave detailed background explanations to follow-up questions
  • pictures are generally better than words

Present Research Project to Corporate Audience

AI Applications

Please find below a few noble and valuable applications of AI:

Conversational AI (symbl.ai)

  • simple: speech-to-text, transcription;
  • harder: generate insights (topics, questions, sentiment), live captioning;
  • hardest: domain intelligence

Pandemical AI (Lengoo blog article)

  • classical approach: predict spread using features such as info about incoming passengers, current infections;
  • deep learning: detect disease from cough (sound processing);
  • develop vaccine with gene sequencing    

Environmental AI 

  • guard against poaching, illegal logging, illegal fishing;
  • AI can be used to extract information from historical data to efficiently plan ranger patrols and strategically allocate resources

Tuesday 23 May 2023

German Driving License (Theory)

Update: Passed the theory exam!

http://www.gettingaroundgermany.info/index.shtml
https://bmdv.bund.de/blaetterkatalog/catalogs/327056/pdf/complete.pdf
https://mycardoeswhat.org/
https://routetogermany.com/drivingingermany/driving-in-germany
https://traffic-rules.com/en/germany

TÜV and Dekra

Class B (passenger car)

    - Maximum authorized mass of 3,5 t
    - Maximum 8 seats + a driver’s seat
    - Allowed trailer mass of max. 750 kg
    - Education and examination in theory and practice
    - Includes the categories: AM (moped and motorcycles) and L (tractor and forklift)

Parking/Stopping

leave gap with cyclists: 1.5m (in build-up area) and 2m (outside build-up area)

leave 5m at intersection, 8m if cycle lane exists10m behind traffic signs

15m gap from bus-stops

3m gap from center line

50m at railroad crossing outside build-up areas and 5m inside build-up areas

stopping distance = reaction  distance + braking distance

    reaction distance = speed * 3 / 10

    braking distance = ( speed * speed ) / ( 10 * 10 )

No stopping: two diagonal lines, more restrictive

No parking: one diagonal line, less restrictive, it's allowed to stop up to three minutes or let in/out passengers or load/unload vehicle

Load

load protrude from rear: 1.5m and 3m (if under 100km), beyond 1m needs clear marking (red light and red reflector)

load protrude from sides: white light at the front and red light at the rear

At a vehicle height under 2.5 m, the load may only protrude to the front by 0.5 m respectively.

maximum permissible total load

maximum permissible axle load

maximum permissible towed load

Vehicle Ownership

Vehicle Registration: Zulassungsbescheinigung Teil 1 (Fahrzeugschein), Stempelplakette (front and rear license plates), Umweltplakette

The registration certificate Part 1 is issued by the registration office in Germany. It contains technical information of the vehicle.
Part 2 of the registration certificate is issued by the dealer or previous owner when the vehicle is purchased. It contains: (1) The name of the owner and, if applicable, of the previous owner; (2) Information concerning the decommissioning and re-registration of the vehicle; (3) Entries concerning technical modifications to the vehicle.

Roadworthiness Test: Hauptuntersuchung; three years after first registration; afterwards every two years; TÜV Plakette (on the rear license plate)

Vehicle Tax

Vehicle Liability Insurance

License Plates (Autoschilder)

Traffic Rule Violation

        A-violation (more serious) and B-violation (less serious) 

        extension of probation by two years

        fines

        license suspension (ban)

        license withdrawal (permanent ban)

        advanced seminar for novice drivers (ASF)

Punkte in Flensburg

driving aptitude seminar (FES): pedagogical and psychological

Speed limit

    50km/hr (within build-up area); 

    30 km/hr (Fahrradstraße), 

    7km/hr (traffic-calming zone), 

 

    100 km/hr (outside build-up area)

    80 km/hr (for cars with trailer outside build-up area)

Miscellaneous

 

Reverse lock: mechanism to prevent accidentally selecting reverse gear

Recuperatuon: regenerative braking

Brake servo unit: power braking

Umwelt zone: emissions sticker (green, yellow, red)

Sunday (and holiday) driving ban: for trucks exceeding 7.5t and trucks with trailer; to ease traffic congestion; to protect environment

The vehicle and its load, taken together, must not be higher than 4m and not wider than 2.55m.

Leave extra gap between vehicle in front if car and trailer together is longer than 7m, to facilitate overtaking by another vehicle.

In Germany, honking and bell ringing is only allowed when it is necessary to warn other road users of a dangerous situation or to prevent a potential accident.

flashing blue light: closed convoy; vehicle is on duty and give way;

flashing amber light: dangers at roadworks or at accident site; exceptionally wide vehicle; large transport proceeding at low speed

orange warning plate: front and rear; to signal dangerous good

Fog lamp: should only be used when your car's main lights won't be enough to make you visible to other road users; front fog lights may be used during the day while visibility is reduced by fog or rain; rear fog lights may be used only if visibility is less than 50m due to fog.

Avoid tailgating: 2-second rule on the highway (i.e. about half the speed in kilometers); 1-second rule in build-up areas

Technicalities

engine stall: sudden stopping of the engine; the ways in which a car can stall are usually down to the driver

engine oil: wear and tear, cooling, clearing, low viscous is better due to lower friction

brake oil: hydraulics

low gear (high engine speed) vs. high gear (low engine revs or low engine speed)

single-track (electric scooter, bicycle, moped, motorcycle) vs. multi-track vehicle (quad, car, truck)

ignition start (normal mode); jump start or push-start or tow-start (dead battery)

Tyre: profile (high profile for higher shock absorption), pressure

Dry steering: puts heavy strain on the steering system, and harms the tyre

Engine states: on, off, idle, stall

Automatic car: 

service brake (foot brake), 

parking brake

P-R-N-D

engine can only switch off in P or N.

engine can only start in P or N

apply foot brake always when selecting gear

the selector level can be changed even when the engine is on/idle

Safety and added measures

Cruise Control: maintains set speed

Adaptive Cruise Control (ACC): independently accelerates to set speed and brakes if necessary

ABS (Anti-lock Braking System): ensure vehicle remains in control (prevent wheel from locking    ) when brakes are applied

Anti-slip Regulation (ASR/Traction Control): ensure vehicle moves forward (prevent wheel from spinning) when accelerator is applied

Lane Keep Assist  System

Automatic Emergency Braking

Tuesday 28 March 2023

'Never Let Me Go' by Kazuo Ishiguro

set in a parallel universe (carers, donors, guardians of secrecy, Madam's Gallery, weekly medical exams, Sales and Exchanges, limited access to the outside world, infertility, evidence in art, organ donation, training, possible, deferral); narration by association; central characters (Kathy, Tommy, Ruth, Miss Lucy, Miss Emily, Miss Geraldine, Madam); 

PART I: Hailsham
Chapter 1: The Tommy Tantrum
Chapter 2: The Lucy Solution
Chapter 3: The Madam Shudder
Chapter 4: The Ruth Encounter
Chapter 5: The Order of the Secret Guard
Chapter 6: Baby, Never Let Me Go
Chapter 7: The Donor Code 
Chapter 8: The Lucy 22
Chapter 9: The Lucy Retreat

PART II: Cottages
Chapter 10: Ruth's Betrayal 
Chapter 11: Kathy's Arousal
Chapter 12: Ruth's Possible
Chapter 13: Chrissie and Roddie's Curiosity
Chapter 14: Chrissie and Roddie's Betrayal
Chapter 15: Tommy's Theories
Chapter 16: Kathy's Helplessness
Chapter 17: Ruth's Amnesia
 
PART III: Carers and Donors
Chapter 18: Pair Rendezvous
Chapter 19: Car Conversation
Chapter 20: Intimacy Increment 
Chapter 21: Madam Meet
Chapter 22: Project Failure
Chapter 23: Tommy Completion

Wednesday 31 August 2022

മനസ്സും കോപവും (സാംജി വടക്കേടം)

കോപം എന്ന വികാരത്തിന്റെ ഗുണങ്ങൾ
  1. മറഞ്ഞിരിക്കുന്ന സ്വഭാവ വൈകല്യങ്ങൾ തെളിഞ്ഞുവരും
  2. സാമൂഹിക നീതി, നന്മ എന്നിവയ്ക്ക് വേണ്ടി പോരാടുവാൻ
  3. മനസ്സിലെ ചിന്തകൾ പ്രകടിപ്പിക്കുവാൻ
  4. സുരക്ഷ; അപകട സാധ്യത തരണം ചെയ്യുവാൻ; ആത്മാഭിമാനം സംരക്ഷിക്കുവാൻ 

കോപിക്കുവാൻ ഉള്ള കാരണങ്ങൾ

  1. മോഹഭംഗം (PhD)
  2. ജോലി (അമിതജോലി)
  3. കുടുംബം (സൗന്ദര്യപ്പിണക്കം)
  4. പരിസ്ഥിതി (മാലിന്യനിർമ്മാർജനം, യാത്രാക്ലേശം)
  5. സാമൂഹിക വ്യവസ്ഥ (സ്ത്രീധനം)
  6. അനാരോഗ്യം (ആർത്തവം)
  7. വാർത്ത
  8. ലഹരിപദാർത്ഥങ്ങൾ
  9. തിരസ്കരണം 
  10. കുറ്റബോധം 

പക്വതയാർന്ന വികാരപ്രകടനം

  1. ആത്മാഭിമാനം
  2. സമചിത്തതയുള്ള ജീവിതം (ശാരീരികം, മാനസികം, ബൗദ്ധികം, ആത്മീകം, കുടുംബം, പ്രകൃതി, സമൂഹം)
  3. ക്രിയാത്മകത (വൈവിധ്യങ്ങളെ തുറന്ന മനസ്സോടെ സ്വീകരിക്കുക)
  4. സമയക്രമീകരണം
    • പണം vs. സമയം (യന്ത്രവൽക്കരണം)
    • ചുമതലപ്പെടുത്തുക (എല്ലാം സ്വയം ചെയ്യണമെന്നുള്ള വാശി ഒഴിവാക്കുക)
  5. നിർദ്ധാരണശൈലി (assertiveness, say NO if necessary)
  6. ആർദ്രത (മനസ്സിലാക്കുക, ആംഗീകരിക്കുക, ക്ഷമിക്കുക)
  7. സംഭാഷണശൈലി
  8. ശ്രവണം
  9. ആംഗ്യഭാഷ (non-verbal communication)
  10. പരിചിന്തനം (retrospection)
  11. പ്രാണായാമം, യോഗ, ധ്യാനം

Sunday 17 July 2022

Love & Love Only

I love you as my own body
I thank God for bringing you into my life

I fall at your feet
I hear your voice
I hold your hands
I caress you gently
I adore your dimple
I massage your legs
I wrap you in my arms
I listen to your heart beat
I kiss you all over the face
I rest my face on your body
I feel comfort in your embrace
I admire the colour of your eyes
I run my hands through your hair

I share my life with you
I partake in your happiness
I join you in your struggles
I aspire to be more than a friend to you

I apologize for hurting you
I hope to improve myself
I try to tame my tongue
I am willing to change

I miss you, milady
I seek you, sweetheart
I bide my time, beloved
I dwell in my thoughts about you, dear
I wait to meet you soon, wife of my youth
I love you, LOVE

Thursday 14 July 2022

Logicomix

This article is a summary of the book Logicomix: An Epic Search for Truth, written by Apostolos Doxiadis and Christos Papadimitriou.

1. Pembroke Lodge: Russel's struggle in comprehending the illogical world (forbidden fruit of knowledge, religious studies, stern grandmother, faith) and finding solace in the logical mathematics (axioms, proofs, theorems, ...)

2. The Sorcerer's Apprentice: Russel breaks loose from the shackles of his strict religious upbringing and becomes an atheist. Russel embarks on a quest for rationality, reasoning, truth and knowledge about the world. However, Russel finds himself in turmoil over the state of mathematics (unproved axioms, circular definitions) and philosophy (disagreement among philosophers). Russel is introduced to logic and works of Boole (Boolean laws). Russel's mission is now to put the house of Mathematics "Queen of the Sciences" in order.

3. Wanderjahr:  Russel goes to the Continent and meets with Gottlob Frege who dreamed of a new logical language to serve as the foundation of mathematics, and (the insane) George Cantor who developed set theory. Russel attends the 1900 ICM Paris in which there are two schools of thought led by Henri Poincaré (proponent of human intuition) and David Hilbert (proponent of mathematical proofs, reasoning and logic).

4. Paradox: Russel discovers the paradox related to definition of sets (the supposed foundation of logic and mathematics), and publishes it in the `Principles of Mathematics.' Russel and Whitehead collaborate on Principia Mathematica, a planned set of three books to create a new foundation of mathematics. The authors struggled a lot while working on the manuscript, and ultimately self-publish the first volume.

5. Entr'acte: Logicians tend to confuse the abstract (Map) with the concrete (Reality).

6. Logico-Philosophical Wars: Wittgenstein begins to study under Russel. At the same time, Europe is reeling under the tension caused by the Great War. Reality confronts Russel. Wittgenstein, due to his trust in natural language, formulates the `Picture Theory,' according to which statements are meaningful only if they can be pictured in the real world. Thus, the statement "There are at least three things in the universe" is meaningless because we can't picture the universe.

7. Incompleteness: Gödel formulates his incompleteness theorem according to which some questions are unanswerable. Russel and Wittgenstein experiment with reforms in the field of education (authoritarian and anti-authoritarian respectively), but ended up as failures in practice. Russel's life (Nazism, Frege's logically-sound anti-semitic rant, WWII) teaches him that there is no royal path to truth. Instead of renouncing the old Trinity composed of Intuition, Emotion and Habit, Russel proposes the new Trinity composed of Responsibility, Justice and knowledge of Good and Evil.  

8. Finale: John von Neumann and later Alan Turing build upon the hitherto works to come up with the Turing machine and the stored-program architecture, and together they pioneer the field of computer science.

"The meaning of the world does not reside in the world." - Wittgenstein thinks to himself.

"All the facts of science aren't enough to understand the world's meaning. For this, you must step outside the world." - says Wittgenstein to Russel.

"Applying formulas is not good enough when you are faced with really hard problems." - Bertrand Russel.

Tuesday 5 April 2022

Automatic Speech Recognition (Part I)

Conversational AI: Automatic Speech Recognition) --> Natural Language Understanding --> Information Retrieval --> Natural Language Generation --> Text-to-Speech

P(W|O)  = argmax_W P(O|W).P(W)
The acoustic model estimates the observation likelihood P(O|W).
The language model estimates the prior probability P(W).

Applications: YouTube closed captioning; Cortana/Siri/Alexa front end; dictation systems

Resources: phonemes, pronunciation dictionary, speech data (labelled for phonemes and words), word vocabulary
https://openslr.org/index.html

Challenges:

  • speech recording: quality and acoustic range of the mic; 
  • background noise;
  • speaker: speed, age and accent of pronunciation; volume of speech (distance between speaker and mic)
  • language-specific constraints: inflections, no writing system;
  • code mixing; multilingual setting; 
  • speed of ASR (esp. in time-critical applications); 
  • task complexity: identify speaker; account for speech-noise; listen continuously or push-to-talk;
  • train ASR for new languages: create resources such as pronouncing dictionary, learn HMM and n-gram probabilities

Components: Acoustic Model (sampling, Fourier transform from time-domain to frequency-domain, pronunciation dictionary, HMM to map phonemes to word), Language Model (n-gram probabilities, smoothing, back-off, linear interpolation)

time-domain: time vs. frequency (pitch) plot
frequency-domain: extract component frequencies with the corresponding amplitudes

Evaluation: Word Error Rate (WER) is a measure of the word-level edit distance in terms of insertions, deletions and substitutions between the reference and the model output.

Vocabulary: humans learn ~30K words by middle age; in case of an open-vocabulary setting, consider all unknown words as the special token <UNK>

HMM: observations (phonemes), states (words), decode the HMM (estimate the best state sequence), train the HMM, evaluate the HMM (estimate the best observation sequence)

International Phonetic Alphabet: list of phones; English has 26 letters to convey the 44 syllable sounds;

CMU Pronunciation Dictionary (CMUdict): mapping between words and phones for North American English; 134000 entries; challenges (names, inflections, numbers)

ASR: CMU Sphinx (2015), Espresso (2019), NeMo, Whisper
TTS: WaveNet (2016), Tacotron (2017)

frequency (pitch)
amplitude (loudness)
waveform;
digitization (sampling, quantization); spectrum computed using Fourier transform; spectrogram; spectral features as vectors (LPC, PLP); phone likelihood estimation (using neural networks);

How to train an HMM?
How to remove background noise?
How to build ASR for different dialects of English?
Do we need a resource that is labelled for phonemes?
How to map speech to phonemes?
Why not build end-to-end ASR systems?

On Construction of the ASR-oriented Indian English Pronunciation Dictionary

https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/asr_language_modeling.html

Monday 7 March 2022

Principles of Computing

 

Theorems or Adages

  • No Free Lunch theorem - there is always a trade-off; originally proposed in the context of search and optimization
  • Black swan theory - theory on seemingly unexpected grand events
  • Hofstadter’s law - "It always takes longer than expected, even if you take into consideration the Hofstadter’s law."
  • Murphy’s law - "Anything that can go wrong will go wrong."
  • Infinite monkey theorem
  • The Chinese Room Argument - [discussed under Linguistics]
  • Pigeonhole principle
  • Moravec’s paradox - While hard tasks such as playing Go, logical reasoning, algebra are easy for AI systems, easy tasks such as moving an object, using common-sense are hard for AI systems.
  • AI effect refers to the tendency for onlookers to dismiss (artificial) intelligence for sheer brute force, number crunching or plain computation. This effect is also observable in magic when the trick gets revealed to the audience.
  • Church-Turing Thesis - Can a Turing machine compute everything that is computable? What is an algorithm? The Church-Turing thesis states that the Turing Machine model is at least as powerful as any computer that can be built in practice. The Church-Turing thesis states that known programming languages are capable of describing everything that could be described as an algorithm. In other words, it says the conventional definition of "algorithm" as that which can be done by executing programs-as-we-know-them is the right definition.
  • Turing-completeness - In computability theory, a system of data-manipulation rules (such as a computer's instruction set, a programming language, or a cellular automaton) is said to be Turing-complete or computationally universal if it can be used to simulate any single-tape Turing machine. So, if somebody says "my new thing is Turing-complete" that means in principle (although often not in practice) it could be used to solve any computation problem. A basic calculator or a scientific calculator is not Turing-complete. Programming languages are similar to those machines (although virtual). They take programs and run them. Most programming languages, conventional and unconventional, such as C, Java, Lisp, Haskell and Prolog, are Turing-complete. The typesetting language LaTeX is also Turing-complete. However, markup languages such as XML and HTML are non-Turing-complete languages. Formal languages such as regular languages and context-free languages are Turing complete. On the other hand, logics such as propositional logic and predicate logic are not Turing complete.
  • AI-complete - The hardest problems in AI (such as vision, natural language understanding, and planning) are AI-complete. In other words, solving an AI-complete problem is equivalent to making computers as intelligent as people.
  • Strong AI vs. Weak AI - Strong AI is general AI (a.k.a. AGI) that can perform any intellectual task that a human being can. Weak AI (a.k.a. narrow AI) is AI that can perform specific intellectual tasks (e.g. IBM Watson).
  • Friendly AI - FAI is general AI that would have a positive impact, rather than a negative effect, on humanity. The closely-related areas are machine ethics and AI takeover.
  • Colossal Disaster - IBM Watson is merely a massive engineering (and marketing) effort comprised of heavy parallelism, huge knowledge base and clusters. The technologies that power Watson include (1) information retrieval and information extraction, (2) computational linguistics (statistical paraphrasing, semantic role labelling), (3) machine learning and (4) knowledge representation and reasoning (temporal reasoning, geospatial reasoning). Watson is expert at playing Jeopardy!. So maybe it is an expert system. But is it an AI system? Roger Schank doesn’t think so because it is yet to display intelligence. If it were an AI system then it should be able to answer the following kinds of questions: (1) between Jim and Betty, who is the better person to lead our new social initiative while creating a more transparent culture, (2) can you create an operational strategy to achieve these revenue objectives, (3) how would you rate the CEO’s speech on employee morale?, (4) how do you think this marketing campaign will impact our customers and prospective customers, or (6) was Jim’s joke funny? These questions pertain to request for opinion, strategy and complex phenomena such as morale, jokes, feeling and emotions.
  • Recommender systems - The technosphere is filled with recommender systems such as Google search, Facebook feed, Twitter feed, YouTube video recommendation and Amazon product recommendation.
Computer Languages
  • General Purpose Programming Language: C (procedural programming, ancient); Java (no pointers, garbage collector, enterprise language, statically typed, Hibernate database application framework, Struts and Spring web application frameworks); Scala (object-oriented + functional programming, statically typed); C++ (object-oriented programming)
  • Scripting Language: Python (dynamically typed, Django web application framework); awk (structured text search and replace)
  • Functional Language: Haskell (functional programming, statically typed)
  • Logical Language: Prolog (logic programming)
  • Query Language: SQL (query rdbms database); XPath (query xml database)
  • Shell Programming Language: bash (run commands from script)
  • Data Language: HTML (exchange data, common in www); XML (exchange data)
  • Modelling Language: RDFS (less expressive modelling); OWL (more expressive modelling)

Saturday 5 February 2022

Deutschlernen Tracker

Dictionary

Grammar

  • YourDailyGerman
  • Accusative prepositions:  für, ohne, durch, um, gegen
  • Dative prepositions: aus, nach, von, bei, zu, mit, ab, seit, außer
  • Two-way prepositions: use the accusative case when there is a position change from point A to point B; use the dative the rest of the time.
    • Ich bin im Kino.
    • Ich gehe ins Kino.
  • The adverb always comes after the verb.
  • Prepositions with Locations: in (inside, as in classroom), an (near, as in beach), auf (on a surface or at a public building such as bank), bei (specific places, as in Zentralmensa) 
    • an der Universität
  • Adjective endings, http://www.nthuleen.com/teach/grammar/adjektivendungenexpl.html
  • Articles
  • Conjunctions
    • Wenn das Wetter gut ist, dann wandern viele Leute nach Heiligenberg. 
    • Weil das Wetter gut ist, wandern viele Leute nach Heiligenberg. 
    • Viele Leute wandern nach Heiligenberg, obwohl das Wetter schlecht ist.

Vocabulary

  • Ankidroid
  • Clozemaster app

Listening 

  • nachrichtenleicht.de
  • SlowGerman (Annik Rubens Podcast)
    • Was wären wir ohne Internet? SG #238
    • Das Schloss Neuschwanstein – SG #237
    • Arm und reich in Deutschland – SG #223
    • Die Hobbys der Deutschen – SG #218
    • Die Uhrzeit – SG #210
    • SG #192: Online-Shopping
    • SG #147: Die Bahn in Deutschland
    • SG #011: Denglisch
    • SG #009: Bus und Bahn 
    • SG #160: Die Volkshochschule

Writing

Reading

Video Series
  • Nicos Weg 
    • Nicos Weg A1
    • Nicos Weg A2
  • Extr@ (German)
    • EXTRA auf Deutsch - Folge 01 | Sams Ankunft
    • EXTRA auf Deutsch - Folge 02 |
    • EXTRA auf Deutsch - Folge 03 | Sam hat ein Date
    • EXTRA auf Deutsch - Folge 04 | Sam sucht einen Job
    • EXTRA auf Deutsch - Folge 05 | Ein Star ist geboren
    • EXTRA auf Deutsch - Folge 06 |
    • EXTRA auf Deutsch - Folge 07 | Der Zwilling
    • EXTRA auf Deutsch - Folge 08 | Die Kusine der Vermieterin
  • Pluspunkt
  • Kendra's Language School

Friday 4 February 2022

Modern DS Software Development

Team Structure

  • Developers
    • Planning: Agile (SCRUM, Kanban)
    • Coding: version control system (svn, git), integrated development environments (Eclipse, PyCharm)
    • Building: makefile, Maven (pom.xml)
    • Testing: Selenium, HP QTP
  • DevOps: Jenkins
  • Operations
    • Deploy: Docker
    • Operate
    • Monitor: Nagios

Web Development

  • browser: Firefox, Chrome, Safari
  • PL: what to do (Java, JavaScript, Python, Ruby, PHP)
  • HTML: what to show
  • CSS: how to show
  • database: how data is stored (MySQL, Oracle, MongoDB)
  • data format: how data is exchanged (JSON, XML)
  • framework: platform or content-management system (Struts/Java, Meteor/JavaScript, Django/Python, Rails/Ruby, WordPress/PHP, Drupal/PHP)
  • front-end: browser, HTML, CSS, PL
  • back-end: server, database
  • client: user of a web application
  • server: where application is stored
  • protocol: how machines communicate (HTTP)
  • cloud computing services
    • compute power, database storage, DBMS, AI services;
    • SaaS (Google Apps); PaaS (Amazon retail, Flipkart retail); IaaS (Google Drive, Amazon Web Services, Microsoft Azure);Network-as-a-Service or NaaS (Asianet ISP); Data-as-a-Service or DaaS

ELK Stack: ElasticSearch (full-text search engine, run ML); Logstash; Kibana (data analysis and visualization)

Databases

  • Relational (SQL) databases
    • have to have a schema
    • poor horizontal scaling (possibly due to tight coupling between tables)
    • e.g. SQL Server, MySQL
  • NoSQL
    • enables better horizontal scaling (adding new servers, partitions)
    • no need to have a schema (good for constantly evolving data)
    • flexible
    • key-value storage (JSON)
    • e.g. MongoDB, Cassandra
Docker
  • facilitate faster application delivery and deployment; ensure portability by separating application from infrastructure; enables a microservice-based architecture
  • wrap OS, dependencies, configuration, directories and files (and not data) into a single lightweight application
  • follows a client-server architecture; relies on Linux virtualization, cgroups, namespaces
  • use Dockerfile to build a Docker image
    • FROM (choose base image), MAINTAINER (for documentation purpose), RUN (execute commands during image building), CMD (execute commands by the running image), ADD (files to image), ENV (define environment variables), VOLUME, WORKDIR (change working directory), USER
  • save images on (public/private) registry, e.g. Docker Hub
  • run images as a Docker container
  • use volumes for persistent data
  • Docker commands
    > docker pull, push
    > docker create, run, start, kill, rm, stop, commit
    > docker images, rmi, ps, logs

Kubernetes

  • container orchestration platform
  • automatically deploy, manage and scale containerized applications
  • k8s cluster services
  • application configuration (app1.yaml file)