homework 1 for KBAI PDF

Preview

Summary

homework 1 for KBAI...

Description

Assignment 1 Naveed Nadjmabadi [email protected] QUESTION 1 1.1 Semantic Network Construction The semantic network that models the Rey-Kylo-Snoke problem is similar to the guards and prisoners problem that was presented in Lecture 3. From the problem description, a semantic network that contains all the necessary state are the location and direction of the shuttle, both the sith (red), and the one jedi (blue), and the corresponding sides that each actor was placed on. The individual identities of either sith, Snoke or Kylo, are irrelevant to solving the problem, as is the presence of Leia aboard the orbital ship.

Figure 1—Semantic Network Representation.

The helicopter represents the shuttle, while the black shaded arrows on either side indicate the location it will be in the next state. Kylo and Snoke are represented as red dots, while Rei is blue. The dotted line in each rectangle is the boundary between Quesh (left) and the Orbital Ship (right). A transition between states is represented as a directional arrow to the next frame, as well as triangle matching the color and direction of the movement of an actor. In the above image, A right-pointing blue triangle indicates that Rei is crossing over from Quesh to the orbital ship. If there is no colored triangle between states, one can assume the shuttle is operating on autopilot. Notice the black arrows on either side of the shuttle change as well. The shuttle is on the right hand-side after the transition, but will be heading back to Quesh.

1

1.2 Application of Generate-and-Test

Figure 2—Generate and Test on Network. For Full-Sized image, please view Appendix 6.1.The red X’s in the top right corner of a frame indicate either SEEN or Invalid. The text above the X indicates what each failure case was.

1.2 Generate and Test After applying Generate and Test on this semantic network, there were 3 Invalid States, as well as 7 SEEN states. There were 19 unique frames before a valid solution was considered, 8 frames deep from the original starting state. In this example, the Tester was smart, and the generator was dumb. The generator would expand one layer at a time, and create potential states that were reachable from the current state. The tester’s responsibility was to then mark all invalid next-states as either SEEN or Failure. For any states that were not discounted, the generator was allowed to proceed one step further. Once the correct solution was found, the application finished processing.

2

QUESTION 2 For the Production System for an Uno Game, the agent was designed to maximize chances of winning by observing the number of the opponents cards available and attempting to play special cards to prevent opponents from winning when their chances of winning exceeds the agent. The AI agent generally favors a strategy of playing cards that would give the agent the most options on subsequent turns. 2.1 Creation of the Production System

Figure 3—Production System for Uno Card game. There are 3 goals: “Rid Cards”, “Prevent Opponent from Winning”, and “choose best color” (for wild-card scenarios)

3

2.2 Observed Performance of AI Agent With these production rules, one match was observed and the output was logged below. Special Cards will be denoted by their full name. Non-special cards will be represented by a shorthand which is the number and the first letter of color. E.g. a yellow 8 will be denoted as “8Y” and a red 5 will be “5R”. Card on Top represents the card at the top of the pile when

Card on Top

Rules Agent Followed

Card played by Agent

8Y

11

5Y

Yellow Skip Card

11

3Y

9Y

1

None, 8B drawn

9R

11

5R

4R

11

0R

2R

10

2B

5B

11, 13

8B

1G

11

6G Victory 

In this round, the agent was victorious, but in many subsequent rounds, it was not. In order to improve the performance of the AI agent, one would need to consider not only how many cards an opponent has left, but how many cards were played and the probability that a given card can be played next. Having this knowledge could determine when the agent should play special cards. Another aspect that this production system does not factor into its strategy is the direction of the rotation. This system would try to undermine either opponent even if the opponent with the higher likelihood of victory was the one that plays before the agent. A better agent would be able to predict which agent has the highest likelihood of success, and then attempt to use special cards to force the opponent to have the sub-optimal hand.

4

QUESTION 3 The General Data Protection Regulation provides a thorough and well-defined set of rules regarding the usage of personal data by a company (aka “controller”) for the intent of personalizing individual user experiences online. These are defined very broadly in the section “Principles relating to the processing of personal data” (Chapter 2, Article 5), where it can be broken down into 4 general themes: transparency, accuracy, minimality, and confidentiality. At its core, the GDPR implies that any personal data that a company processes, e.g. the “collection, recording, organisation …” (Chapter 1, Article 4) must be made only after explicit consent from the user, and that the user has certain guarantees about that data for the entirety of time that data can still be linked to the user. Among them, the user has the “right to be forgotten (erasure)”, “the right of restriction of processing”, the “right to rectification”, and the right to revoke access. In addition, the company cannot collect information from users under the age of 13, and until the data subject is 16, the company must get formal approval from the data subjects’ legal guardian. (Chapter 2, Article 8). For AI practitioners, this poses various constraints on the data collected and how it may be used to personalize user experiences. For modern applications of Machine Learning, AI practitioners must take care to pseudonomize the data that it collects from a user. This poses non-trivial engineering challenges to ensure that data is encrypted in-transit and at-rest (Chapter 4, Section 2, Article 32). In addition, one must be cognisant to collect only the minimum data necessary to build the AI. This requires foresight on the kinds of data features one would need to build effective ML models. Prior to this, many practitioners were encouraged to act ethically, but had no restrictions on what data they could collect after they received a general consent. The GDPR explicitly states that “personal data shall be collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes.” (Chapter 2, Article 5) Finally, an AI practitioner must be aware that the consumer has the right to erasure (a.k.a the right to be forgotten), which means the deletion of all records identifying that data subject. Having less data can impact the efficiency of the models that are created.

5

3.1 Case Analysis of Personalization Enabled Device A device that depends heavily on personalization in both its functional purpose and its business model is the Ring Home Security / Video Doorbell sold by Amazon. This device depends heavily on personalization because it collects a data-subject’s biometric data (facial recognition) in order to recognize the owner(s) of the device. In addition, it records any activity that occurs in a specified geographical area and uploads this to a 3rd party controller (AWS services). The other notable competitor to Ring is Nest Hello (owned by Google). To adapt to GDPR regulations and be sold in Europe, Ring has created their own web page where they outline how they are GDPR compliant (). In their FAQ, they specify that, if the user has already consented, videos are passed to a 3rd party processor (Amazon AWS cloud servers) where the video is transcoded and stored, otherwise it will be deleted. If the user has not consented to the storage of the video, but has explicitly authorized a 3rd party app (“Neighbors”), the video will be stored only while it is transcoded and delivered to the app. The user is given the right to access and the right to erasure explicitly via controls in the app. If they choose, they are allowed to delete any and all recordings saved. Ring will not collect any data from the recordings unless explicitly authorized by the user or if the user has made the data publicly available via the Neighbors app, which is an app they also own. If the users post to the Neighbors app, one can argue from the following passage that the data is now considered public and therefore not subject to GDPR restrictions: “the processing of ... biometric data … shall be prohibited … [but] shall not apply if … processing relates to personal data which are manifestly made public by the data subject” (Chapter 2, Article 9). Based on the amount of personal information that is required for this device to operate, it is impossible to allow users in the European Economic Area to use this tool in its current form without waiving their GDPR rights. This device cannot function without the collection and processing of the subject’s biometric data. Unless all data storage and processing were to occur locally, it is not possible to use this device without forfeiting GDPR rights.

6

QUESTION 4 While IBM’s Watson banked significantly higher than both Ken Jennings and Brad Rutter in Jeopardy, Watson is remarkably less intelligent as a cognitive agent than either of the two other contestants. While Watson demonstrated novel strategies for certain aspects of the game, the vast majority of Watson’s wins can be attributed to the “buzzer factor” more so than superior intelligence. To provide a working definition of intelligence, we begin from the “Assumptions of Cognitive Agents” (KBAI, Lecture 6). Cognitive Systems are generally composed of 3 parts: deliberation, reaction, and metacognition. Cognitive agents -- both natural and artificial -- are goal oriented, take actions to reach their goals, and use knowledge to make decisions about which actions to take. They operate in rich, dynamic environments, learn from their experiences, and most importantly (emphasis mine), adapt to their environment. This ability to adapt and learn new strategies to optimize goal outcomes is critical to the success of a cognitive agent. Agents that can adapt faster, therefore, are more intelligent. If we accept this definition of intelligence, how can one prove that the human contestants are better at adapting? Let us consider the largest disparity between humans and Watson -- their reaction time. From the the web blog of Ray Kurzewil, “the accepted figure for mean simple reaction times for college-age individuals for light stimuli is about 190 ms (0.19 sec)” (Kurzweil, 2011). The article then states “[Watson’s] reaction time is just five to ten milliseconds… [which is] a 38:1 advantage”. Furthermore, the 190ms estimate is for a college aged individual, while Wikipedia shows that both human contestants were over the age of 30 during the taping of the episodes. Studies have shown (Tun & Lachman, 2009) that mean performance reaction time decreases with age, which means that 190ms is an overly optimistic estimation of a human’s ability to press the buzzer in time. In the Kurzweil blog, one of Watson’s creators, David Ferrucci, made the counterclaim that human operators could “time the buzz”, anticipating when the light would turn on and therefore have a headstart on Watson. But in practicality, it seems unlikely that one could time the buzz and also formulate a response to the question before the 5-10ms it would take Watson to answer. Based on the j-archive statistical averages of Watson’s 37 total responses to Ken Jennings 15 and Brad Rutten’s 13, it seems to corroborate that

7

neither human contestant was able to use the “time the buzz” technique in their favor. The effect of the Buzzer Factor further becomes obvious in all of the Final Jeopardy rounds, when the contestants were n  ot being evaluated on their reaction times. Ken Jennings and Brad Rutter got both questions correct, whereas Watson only got one. Furthermore, Watson’s response to the final Jeopardy on February 15, 2011 is surprising. Given the category of “U.S. cities”, it responded with Toronto (a city in Canada), which shouldn’t have been considered to begin with since it's not part of the category. Watson’s wager was almost $1000 even though it didn’t know the answer. An intelligent agent, given that it has insufficient data for a meaningful answer, would be expected to wager $0 to minimize losses. Consequently, the human cognitive agents were aware that this was their only opportunity to get even with Watson, and placed all-or-nothing bets on their responses. Faced with the predicament of not being able to beat Watson in a reaction-time based contest, both human contestants learned that they needed to change their strategy to remain competitive, and performed optimally in those scenarios. One notable instance was in game 3 at the 11:19 mark, when Ken Jennings selected a square that turned out to be a daily double, he was asked how much he wanted to wager. He responded with “I have two options: either unplug Watson or bet it all!” By betting his entire amount, he was able to surpass Watson at that point in the game. More importantly, by musing that the only way to beat his opponent was outside the means of the game, he demonstrated a human agent’s natural ability for metacognition, or learning about learning. He again demonstrates this ability again at 18:01 in the 3rd episode of the competition by adding a joke for his Final Jeopardy response: “(I for one welcome our new computer overlords)”. As a cognitive system, he is able to reflect upon how the game is progressing and his own performance in it; a skill Watson did not, and furthermore, could  not demonstrate. While Watson’s performance in Jeopardy was impressive, it was just that -- a performance. In the end, the humans adapted quickly to deal with the significant timing handicap, and performed admirably in overwhelming odds. If given a level playing field, it’s clear they would be more than a match for Watson.

8

5 REFERENCES 1. REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL. (2016, April 27). Retrieved January 26, 2020, from https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:3201 6R0679&from=EN#d1e40-1-1 2. Privacy. (n.d.). Retrieved January 26, 2020, from

https://eu.ring.com/pages/privacy 3. Angelica, A. D. (2011, February 18). Kurzweil Accelerating intelligence. Retrieved January 26, 2020, from https://www.kurzweilai.net/the-buzzer-factor-did-watson-have-an-unfair -advantage 4. Analysis of Watson's Strategies for Playing Jeopardy! IBM TJ Watson Research Center, Yorktown Heights, NY 10598 USA, May 2013, arxiv.org/ftp/arxiv/papers/1402/1402.0571.pdf 5. Internet Archive,

archive.org/details/Jeopardy.2011.02.The.IBM.Challenge/Jeopardy.2011.0 2.16.The.IBM.Challenge.Day.3.HDTV.XviD-FQM.avi. 6. Tun, P. A., & Lachman, M. E. (2008, September). Age differences in reaction time and attention in a national telephone sample of adults: education, sex, and task complexity matter. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2586814/

7. Player statistics - Watson. (n.d.). Retrieved January 25, 2020, from http://www.j-archive.com/showplayerstats.php?player_id=7208 8. Player statistics - Ken Jennings. (n.d.). Retrieved January 25, 2020, from http://www.j-archive.com/showplayerstats.php?player_id=13086 9. Player statistics - Brad Rutter. (n.d.). Retrieved January 25, 2020, from http://www.j-archive.com/showplayerstats.php?player_id=13085 6 APPENDICES 6.1 Generate and Test Image (Viewable on next page)

9

10...