M.S. Thesis Defense

Neuroevolution-Based Inverse Reinforcement Learning

Karan K. Budhraja

9:00am Wednesday, 2 December 2015, ITE 346

Motivated by such learning in nature, the problem of Learning from Demonstration is targeted at learning to perform tasks based on observed examples. One of the approaches to Learning from Demonstration is Inverse Reinforcement Learning, in which actions are observed to infer rewards. This work combines a feature based state evaluation approach to Inverse Reinforcement Learning with neuroevolution, a paradigm for modifying neural networks based on their performance on a given task. Neural networks are used to learn from a demonstrated expert policy and are evolved to generate a policy similar to the demonstration. The algorithm is discussed and evaluated against competitive feature-based Inverse Reinforcement Learning approaches. At the cost of execution time, neural networks allow for non-linear combinations of features in state evaluations. These valuations may correspond to state value or state reward. This results in better correspondence to observed examples as opposed to using linear combinations.

This work also extends existing work on Bayesian Non-Parametric Feature construction for Inverse Reinforcement Learning by using non-linear combinations of intermediate data to improve performance. The algorithm is observed to be specifically suitable for a linearly solvable non-deterministic Markov Decision Processes in which multiple rewards are sparsely scattered in state space. Performance of the algorithm is shown to be limited by parameters used, implying adjustable capability. A conclusive performance hierarchy between evaluated algorithms is constructed.

Committee: Drs. Tim Oates, Cynthia Matuszek and Tim Finin

M.S. Thesis Defense

Neuroevolution-Based Inverse Reinforcement Learning

Karan K. Budhraja

9:00am Wednesday, 2 December 2015, ITE 346

Motivated by such learning in nature, the problem of Learning from Demonstration is targeted at learning to perform tasks based on observed examples. One of the approaches to Learning from Demonstration is Inverse Reinforcement Learning, in which actions are observed to infer rewards. This work combines a feature based state evaluation approach to Inverse Reinforcement Learning with neuroevolution, a paradigm for modifying neural networks based on their performance on a given task. Neural networks are used to learn from a demonstrated expert policy and are evolved to generate a policy similar to the demonstration. The algorithm is discussed and evaluated against competitive feature-based Inverse Reinforcement Learning approaches. At the cost of execution time, neural networks allow for non-linear combinations of features in state evaluations. These valuations may correspond to state value or state reward. This results in better correspondence to observed examples as opposed to using linear combinations.

This work also extends existing work on Bayesian Non-Parametric Feature construction for Inverse Reinforcement Learning by using non-linear combinations of intermediate data to improve performance. The algorithm is observed to be specifically suitable for a linearly solvable non-deterministic Markov Decision Processes in which multiple rewards are sparsely scattered in state space. Performance of the algorithm is shown to be limited by parameters used, implying adjustable capability. A conclusive performance hierarchy between evaluated algorithms is constructed.

Committee: Drs. Tim Oates, Cynthia Matuszek and Tim Finin

Ph.D. Dissertation Defense
Computer Science and Electrical Engineering
University of Maryland, Baltimore County

Rapid Plan Adaptation Through Offline
Analysis of Potential Plan Disruptors

Robert H. Holder, III

9:00am Wednesday, 9 December 2015, ITE 325b

Computing solutions to intractable planning problems is particularly problematic in dynamic, real-time domains. For example, visitation planning problems, such as a delivery truck that must deliver packages to various locations, can be mapped to a Traveling Salesman Problem (TSP). The TSP is an NP-complete problem, requiring planners to use heuristics to find solutions to any significantly large problem instance, and can require a lengthy amount of time. Planners that solve the dynamic variant, the Dynamic Traveling Salesman Problem (DTSP), calculate an efficient route to visit a set of potentially changing locations. When a new location becomes known, DTSP planners typically use heuristics to add the new locations to the previously computed route. Depending on the placement and quantity of these new locations, the efficiency of this adapted, approximated solution can vary significantly. Solving a DTSP in real time thus requires choosing between a TSP planner, which produces a relatively good but slowly generated solution, and a DTSP planner, which produces a less optimal solution relatively quickly.

Instead of quickly generating approximate solutions or slowly generating better solutions at runtime, this dissertation introduces an alternate approach of precomputing a library of high-quality solutions prior to runtime. One could imagine a library containing a high-quality solution for every potential problem instance consisting of potential new locations, but this approach obviously does not scale with increasing problem complexity. Because complex domains preclude creating a comprehensive library, I instead choose a subset of all possible plans to include. Strategic plan selection will ensure that the library contains appropriate plans for future scenarios.

Committee: Drs. Marie desJardins (co-chair), Tim Finin (co-chair), Tim Oates, Donald Miner, R. Scott Cost

Ph.D. Dissertation Defense
Computer Science and Electrical Engineering
University of Maryland, Baltimore County

Rapid Plan Adaptation Through Offline
Analysis of Potential Plan Disruptors

Robert H. Holder, III

9:00am Wednesday, 9 December 2015, ITE 325b

Computing solutions to intractable planning problems is particularly problematic in dynamic, real-time domains. For example, visitation planning problems, such as a delivery truck that must deliver packages to various locations, can be mapped to a Traveling Salesman Problem (TSP). The TSP is an NP-complete problem, requiring planners to use heuristics to find solutions to any significantly large problem instance, and can require a lengthy amount of time. Planners that solve the dynamic variant, the Dynamic Traveling Salesman Problem (DTSP), calculate an efficient route to visit a set of potentially changing locations. When a new location becomes known, DTSP planners typically use heuristics to add the new locations to the previously computed route. Depending on the placement and quantity of these new locations, the efficiency of this adapted, approximated solution can vary significantly. Solving a DTSP in real time thus requires choosing between a TSP planner, which produces a relatively good but slowly generated solution, and a DTSP planner, which produces a less optimal solution relatively quickly.

Instead of quickly generating approximate solutions or slowly generating better solutions at runtime, this dissertation introduces an alternate approach of precomputing a library of high-quality solutions prior to runtime. One could imagine a library containing a high-quality solution for every potential problem instance consisting of potential new locations, but this approach obviously does not scale with increasing problem complexity. Because complex domains preclude creating a comprehensive library, I instead choose a subset of all possible plans to include. Strategic plan selection will ensure that the library contains appropriate plans for future scenarios.

Committee: Drs. Marie desJardins (co-chair), Tim Finin (co-chair), Tim Oates, Donald Miner, R. Scott Cost

UMBC Department of Information Systems

Is Bigger Better? Comparing User Generated Passwords on
3×3 vs. 4×4 Grid Sizes for Android’s Pattern Unlock

Adam Aviv, USNA

1:00-2:00pm Tuesday, 1 December 2015, ITE 459

Android’s graphical authentication mechanism requires users to unlock their devices by “drawing” a pattern that connects a sequence of contact points arranged in a 3×3 grid. Prior studies have shown that human-generated patterns are far less complex than one would desire; large portions can be trivially guessed with sufficient training. Custom modifications to Android, such as CyanogenMod, offer ways to increase the grid size beyond 3×3, and in this paper we ask the question: Does increasing the grid size increase the security of human-generated patterns?

To answer this question, we conducted two large studies, one in-lab and one online, collecting 934 total 3×3 patterns and 504 4×4 patterns. Analysis shows that for both 3×3 and 4×4 patterns, there is a high incidence of repeated patterns and symmetric pairs (patterns that derive from others based on a sequence of flips and rotations). Further, many of the 4×4 patterns are similar versions of 3×3 patterns distributed over the larger grid space. Leveraging this information, we developed the most advanced guessing algorithm in this space, and we find that guessing the first 20% (0.2) of patterns for both 3×3 and 4×4 can be done as efficiently as guessing a random 2-digit PIN. Guessing larger portions of 4×4 patterns (0.5), however, requires 2-bits more entropy than guessing the same ratio of 3×3 patterns, but the entropy is still on the order of cracking random 3-digit PINs. These results suggest that while there may be some benefit to expanding the grid size to 4×4, the majority of patterns will remain trivially guessable and insecure against broad guessing attacks.

Adam J. Aviv is an Assistant Professor of Computer Science at the United States Naval Academy, receiving his Ph.D. from the University of Pennsylvania under the advisement of Jonathan Smith and Matt Blaze. He has varied research interests including in system and network security, applied cryptography, smartphone security, and more recently in the area of usable security with a focus on mobile devices.

UMBC Department of Information Systems

Is Bigger Better? Comparing User Generated Passwords on
3×3 vs. 4×4 Grid Sizes for Android’s Pattern Unlock

Adam Aviv, USNA

1:00-2:00pm Tuesday, 1 December 2015, ITE 459

Android’s graphical authentication mechanism requires users to unlock their devices by “drawing” a pattern that connects a sequence of contact points arranged in a 3×3 grid. Prior studies have shown that human-generated patterns are far less complex than one would desire; large portions can be trivially guessed with sufficient training. Custom modifications to Android, such as CyanogenMod, offer ways to increase the grid size beyond 3×3, and in this paper we ask the question: Does increasing the grid size increase the security of human-generated patterns?

To answer this question, we conducted two large studies, one in-lab and one online, collecting 934 total 3×3 patterns and 504 4×4 patterns. Analysis shows that for both 3×3 and 4×4 patterns, there is a high incidence of repeated patterns and symmetric pairs (patterns that derive from others based on a sequence of flips and rotations). Further, many of the 4×4 patterns are similar versions of 3×3 patterns distributed over the larger grid space. Leveraging this information, we developed the most advanced guessing algorithm in this space, and we find that guessing the first 20% (0.2) of patterns for both 3×3 and 4×4 can be done as efficiently as guessing a random 2-digit PIN. Guessing larger portions of 4×4 patterns (0.5), however, requires 2-bits more entropy than guessing the same ratio of 3×3 patterns, but the entropy is still on the order of cracking random 3-digit PINs. These results suggest that while there may be some benefit to expanding the grid size to 4×4, the majority of patterns will remain trivially guessable and insecure against broad guessing attacks.

Adam J. Aviv is an Assistant Professor of Computer Science at the United States Naval Academy, receiving his Ph.D. from the University of Pennsylvania under the advisement of Jonathan Smith and Matt Blaze. He has varied research interests including in system and network security, applied cryptography, smartphone security, and more recently in the area of usable security with a focus on mobile devices.

The UMBC Cyber Defense Lab presents

Security Review of the MyUMBC Mobile App

Mikhail Aleksander, Enis Golaszewski, Gavin Lebo and Daniel Whitt

11:15am-12:30pm Friday, 20 November 2015, ITE 231

Our team will present preliminary findings and lead an informal discussion on its project to carry out a security review of new custom software for mobile devices in the UMBC enterprise. Using Highpoint, this custom software allows users to connect from IOS and Android mobile devices to application services including Peoplesoft (registration and administrative functions), Blackboard (instructional support), and Cashnet (campus financial transactions). Focusing on the custom software, the review includes an adversarial model, summary of the data and resources to be protected, analysis of the system design and architecture, and static and dynamic analysis of the source code using a variety of tools. Among other questions, the review addresses the following: What are potential vulnerabilities? How might an adversary exploit these vulnerabilities? What attacks are possible, how difficult would it be to carry out such attacks, what would their consequences be, and what is the risk of such attacks? Are appropriate cryptography and protocols used, are they used appropriately, and are the key lengths appropriate? Is the key management sound, and where are keys stored? Does the design and implementation follow best practices? The final report will include constructive recommendations.

Mikhail Aleksander, Enis Golaszewski, Gavin Lebo, and Daniel Whitt are students in Dr. Sherman’s CMSC-491/691 Cybersecurity Research class of the NSF-funded INSuRE project.  Aleksander, Golaszewski, and Lebo are BS students in computer science; Whitt is a MPS student in Cyber. Lebo and Whitt are also SFS Scholars.

Host: Alan T. Sherman, Sorry, you need javascript to view this email address.

MS Defense
UMBC Computer Science and Electrical Engineering

Distance Adaptation of Diffuse Reflectance
and Subsurface Scattering

Elizabeth Baumel

1:30pm Friday, November 20, ITE 352, UMBC

Objects in the world around us are made of a myriad of materials, both metallic and non-metallic. Most non-metallic materials scatter light in varying amounts within their surfaces, giving softer, more saturated diffuse colors and softer-edged shadows. This effect, subsurface scattering, is important to make translucent objects look realistic. Non-metallic objects that are opaque also scatter light, just at a very small distance. These non-metallic materials may look somewhat translucent at very close viewing distances, but from farther away they exhibit a more opaque, but still soft diffuse appearance. To shade these objects realistically from all distances, a method is needed to model subsurface scattering effects at close ranges and to smoothly transition to a soft diffuse reflection at larger viewing distances. We present a method that takes advantage of graphics processor texture filtering hardware to linearly filter maps that encode diffuse reflection and translucency information and to interpolate between a close-range subsurface scattering effect and a long-range reflectance function.

Committee: Drs. Marc Olano (Advisor, Chair), Penny Rheingans, Jian Chen

UMBC’s Information Systems Security Association Chapter and Cyber Scholars & Affiliates Program will host a panel on Women and Leadership in IT followed by hors d’oeuvres and networking with the panelists and representatives from Northrop Grumman.  The event will take place from 5:00 to 6:30pm on Wednesday, 18 November 2015 in room 312 of the University Center at UMBC. Panelists include:

• Deborah Bonanni: Former Chief of Staff of NSA & VP of Intelligent Decisions, Inc.
• Diane Howard: VP of Cyber Operations of Northrop Grumman
• Belinda Coleman: President/CEO the Coleman Group Inc.
• Brenda Martineau : Organizational Leadership & Management Skill Community Director of NSA
• Jennifer R. Walker : President/CEO Resolute Technologies, LLC

Everyone is welcome. See the event announcement  for more information and to optionally RSVP.

UMBC ACM Tech Talk

Giving Successful Technical Presentations
Prof. Cynthia Matuszek, UMBC

2:00pm Wednesday 18 November 2015, ITE325

Giving talks is one of the core tasks of a researcher. Technical presentations are how we accomplish some of our most important tasks: talks are the first step in getting other people excited about our work, getting suggestions and feedback, teaching, and applying for jobs and grants. Nonetheless, the art and science of giving a really good technical talk is one we are more likely to leave to chance than to deliberately train in. Not only does this mean we aren’t accomplishing everything we could with our presentations; we’re missing a chance to distinguish ourselves by improving a comparatively rare — but learnable — skill.

In this talk, I will describe the idea of the “culture of conveying information,” and give a number of specific suggestions for improving technical talks — including tools, rules of thumb, social conventions, and suggestions for making your talks engaging, informative, and memorable.

Cynthia Matuszek is an Assistant Professor at the University of Maryland, Baltimore County’s Computer Science and Electrical Engineering department where she heads the Interactive Robotics and Language lab. She completed her Ph.D. at the University of Washington in 2014, where she was a member of both the Robotics and State Estimation lab and the Language, Interaction, and Learning group. She is published in the areas of artificial intelligence, robotics, ubiquitous computing, and human-robot interaction. Her research interests include human-robot interaction, natural language processing, and machine learning.