James Fahey

When Spaceballs, the epic comic parody based on Star Wars, came out in 1987, a little known and now forgotten film critic announced that Spaceballs signaled the demise of science fiction film as a genre.  His logic was as follows: The appearance of a comic parody 10 years after the original film release could only mean that science fiction had little new to offer.  In short, the genre was doomed, strangled in a dead end by a dearth of original material.

Yet here we are almost 30 years later, and last year (2015) over 30 feature length science fiction films were released into the American film market.  True, some of them (eg: Jupiter Ascending, Pixels, and Hot Tub Time Machine 2) were horrible – or simply silly – with little social, technical, creative, or artistic merit.  Others (eg: Jurassic World and Terminator Genesis) could only be explained as money machines drawing on the reboot of established SciFi brands.  SciFi films for the summer of 2016 had similar problems; Star Trek: Beyond, Independence Day: Resurgence, and Ghostbusters are all largely considered flops despite famous heritage.  Looking only at these films, one might agree with the critic’s accusation that science fiction was out of material.

But lets take a closer look, especially at 2015 for which we have a full year of films – including the important Thanksgiving and year-end moviegoing weekends.  Several science fiction movies released in 2015 were well worth seeing, and still others were seriously good movies.

So what makes a “seriously good” science fiction movie?

+ Craftsmanship: A seriously good movie must meet high standards for story, acting, script, special effects, and other basic mechanics of a movie.

+ More than Entertainment:  A seriously good movie needs to be thought provoking.  In particular, it should look at social, political, or scientific issues.  Alternatively, the movie can inspire us to improve the world we live in – or will live in.

+ Plausibility: Finally, viewers must walk away from a seriously good movie believing that what they have just seen is possible.

Unfortunately, the plausibility requirement automatically excludes some long time favorite SciFi movies from consideration as “seriously good,” but movies that get all the craftsmanship elements right are well worth seeing and we have plenty of them in our sample of 2015 movies.  Star Wars VII: The Force Awakens and Mad Max: Fury Road fall into this category.  These movies are both entertaining and engrossing; they can be as much fun as a two hour roller coaster ride.

2015 saw several commendable efforts that went beyond entertainment.  For example, both Tomorrowland and Hunger Games benefit from having underlying themes that are more important than plot and almost as important as the special effects.  Both movies are arguably more than raw entertainment.  Both, however, lack plausibility, both succumb to Hollywood desire for flash, and Tomorrowland additionally suffers from weak craftsmanship – predictability and a somewhat obvious lecturing tone.  Besides, it doesn’t take long to find other 2015 SciFi movies that are worth seeing – depending on your tolerance for foul language (Chappie) or your patience with familiar concepts (Self/less).

Of the 2015 crop of SciFi movies, the ones that meet all the criteria and qualify as “seriously good” are Ex Machina and The Martian

Tension builds steadily in Ex Machina, but you won’t find any explosions and little gore in this low key film.  The movie is sexy but shows no sex.  The special effects are seamless and (along with the sexiness) serve the tragic plot line and the underlying messaging.  If you haven’t seen Ex Machina, put it at the top of your list.

The Martian, unlike Ex Machina, wants very badly to be a blockbuster film, and in the end it caves to the shallow demands of Hollywood to serve up a thoroughly unplausable climax.  Up to the very end, however, The Martian largely adheres to good science, good film craftsmanship, plausibility, and thoroughly entertaining storytelling.  If you’re a science fiction fan, you’ve undoubtedly already seen The Martian, but it’s worth seeing twice.  If you aren’t a science fiction fan, you’re still likely to enjoy The Martian.  Either way, just close your eyes at the end.

With movies like Ex Machina and The Martian, science fiction has shown that it is far from dead as a film genre.  If anything, SciFi is gaining strength.

As humankind becomes more confident of the future and of our own ability to shape it, good science fiction helps us imagine the science, technology, and social changes that are possible.  We can investigate both the outcomes that we want to avoid and the opportunities that we want to embrace.  Science fiction is far from dead.  More people embrace seriously good science fiction than ever before.  We just have to hope that Hollywood – or someone – will make the movies.

For those who have already played Pokémon GO and understand Augmented Reality (AR), you may skip to the next paragraph.  For those who have not or do not, Pokémon GO features real time insertion of animated images (creatures called Pokémon) into a live video stream that comes through the camera lens on your smart phone.  Point your camera lens at a the sidewalk in front of you and, if the software expects a Pokémon to be in that geography, you’ll see it on your screen even though you don’t see it when you look directly at the sidewalk.

Outside of Pokémon GO, you can find a number of examples of excellent AR implementations on contemporary TV, especially in sports broadcasts.  That yellow first down line you see in American football games?  Yup, that’s great AR.  It moves with the field when the camera pans, tilts, or zooms and it disappears behind players when they walk over it.  It really does look like there’s a yellow line painted on the field.  But if you think about it, you know the line is not really there.  No one in the stadium can see it – only people watching the live TV broadcast.

You know those ads on the wall behind home plate during The World Series?  Yup.  Those, too, are artificially inserted.  In fact, depending on where you are in the world, you’ll see different ads.  And they exhibit the same characteristics as the yellow first down line: they stick with the field when the camera pans, tilts, or zooms and they disappear behind anyone who walks in front of them.

The math and the video technology behind those two examples of AR would blow your mind.   Watch for more AR this summer during the Olympics.  You’ll notice that all these AR implementations exhibit the same three characteristics (in order of technical difficulty):

1. A graphic image, sometimes animated, is inserted into a live video stream.

2. The image seems to move with its “real world” surroundings, whether the real world surroundings consists of an American football field, a wall in a baseball stadium, or (in the Olympics) the bottom of a pool.

3. The inserted image disappears behind any object that passes between the camera lens and the place where the inserted image is supposed to be – including batters, catchers, and umpires in a baseball game or referees and players in a football game.

But what about Pokémon GO?

The AR software used in Pokémon GO is very cool and exciting, but it’s also pretty basic.  It only really manages the first of the three features mentioned above, and it’s the easiest one.  It also partially manages the second characteristic.  The Pokémon are inserted into the video stream when you point your camera in the direction of the Pokémon, but the Pokémon doesn’t move with the “real world” very smoothly.  If you tilt, pan, or zoom your camera, the Pokémon moves around fairly wildly and doesn’t give a very good illusion of actually being part of the “real world” surrounding it.

The third and most difficult characteristic of AR, of course, isn’t implemented at all in Pokémon GO.  If you find a Pokémon and, while viewing it through your camera lens, pass your hand between the lens and the Pokémon, the “real world” surrounding the Pokémon will disappear behind your hand, but the Pokémon will not (see the accompanying image).  The same thing applies if a friend, a car, or a passing dog comes between your phone and the Pokémon: the world disappears behind the new object but the Pokémon does not.

There’s no doubt about it: Pokémon GO has brought augmented reality into everyday life and made it fun for millions of people.  As video processing power on mobile devices improves and more games take advantage of AR (which they will) the AR implementation will improve and we’ll see better integration of the artificial images with the surrounding environment – and they’ll begin to disappear behind intervening objects.  For now, although the Pokémon GO implementation is far from ideal, “Progress over Perfection.”

While both augmented reality (AR) and virtual reality (VR) may use computing power, they use very different applications to achieve their ends.

Ok. Time to get the boring stuff out of the way…

First, a quick note on the distinction between AR and VR.

VR is the complete replacement of the real world with an artificial world.  At this time, VR usually replaces only visual and auditory input, but VR fans and businesses are increasingly incorporating artificial stimuli for other senses, especially balance (motion detection), touch, and smell, to make ever more complete virtual realities.

AR, on the other hand, is the artificial, seamless, and dynamic integration of new content into, or removal of existing content from, perceptions of the real world.  AR is most commonly seen in sports broadcasts, such as the yellow first down line in American football games, national flags in the lanes of swimmers and runners in the Olympics, and advertisements on the wall behind home plate in baseball games.  These augmentations of reality are so convincing that most TV viewers do not realize that they are artificially inserted.

Now for a small surprise…

Considering these two descriptions, one might assume that VR is much harder to implement than AR.  After all, VR has to replace all that visual and auditory input.  In fact, however, AR requires much more sophisticated programming and consumes more computing horsepower.

It turns out that recognizing and tracking a range of objects in the real world requires powerful, very fast computing.  The vast majority of computing power in augmented reality is consumed in identifying and tracking reality.  The insertion and removal of content, on the other hand, is relatively simple (other than occlusion, which is hard and discussed later).

Seeing something is not the same as recognizing it…

Human vision can identify all kinds of complex objects, distinguishing objects with only small apparent differences.  It’s useful, for example, to be able to tell your spouse from everyone else of the same gender with the same color hair, skin, and eyes (and let’s face it, there are a lot of people out there who are the same in those respects).

Computer vision still has a hard time even recognizing a large number of random, individual objects.  Specialized software exists for facial recognition and is used routinely by Facebook, iPhoto, and other applications, but to work reliably the camera angles and lighting have to be consistent.  Artificial intelligence (AI) that can generalize facial recognition skills into recognizing everything from sofas to automobiles and distinguishing between different kinds of sofas and automobiles has not yet made it into the public domain.

For now, AR must, therefore, settle for recognizing a few, clearly defined and pre-programmed objects: lines on a football field, or the blank green box behind home plate in a baseball stadium, for example.

As a further complication, with augmented reality, the developer has to allow for the nearly infinite array of variables that the “real world” could throw at his program, from shifting light, to varying camera angles, to falling rain, to a van partially obscuring a sign or the front of a building (making it unrecognizable to the AR application).

This is the point at which virtual reality programming becomes much easier than augmented reality.  In virtual reality, everything is known so all changes are understood.  If a van partially obscures the front of a building in a virtual world, the program knows about it because the van and the front of the building are both part of the virtual world created by the program.  If it starts to rain, or it snows, or a thick blanket of fog changes objects from clearly defined images into ghostly shadows, the program still knows what they are and how they will behave because the weather and the objects are part of the program.

Even the apparently random movements of a player controlled object in a virtual world are completely known to the program because, although the commands that result in the random movement come from an external source (the human), the program makes the changes in the player controlled object in the virtual world and thus knows how the object is changing or moving in the virtual world.

Believe it or not, it gets harder…

One of the big problems in augmented reality is determining whether a real world object is in front of or behind an augmented reality object.  For example, let’s assume you are watching a digital zombie walking down a sidewalk in a crowded city using a smartphone to do the appropriate digital insertion of the zombie.  Some of the (real) pedestrians will be behind the zombie and some will be in front of it.  Of course, the zombie must obscure the pedestrians behind it and the pedestrians in front of the zombie must obscure the zombie.  This is known as occlusion: objects in front hiding (or partially hiding) objects in back.

Right now, smartphones are far too dumb to handle occlusion, and, if the AR application chooses the wrong object to put in front, the results are visually very disturbing.  Therefore, AR apps on smartphones are learning the tracking piece, but have yet to make a serious attempt at occlusion, which means all of today’s AR digital insertions on smartphones and tablets float on top of the real world instead of integrating into it.

In the case of virtual reality, again, the program knows where each object is as well as the viewing angle, so, while the graphic rendering involved in having one object disappear behind another object and then reappear on the other side may not be easy, the harder process of identifying which object is in front of which is already solved.

Another big problem for AR is camera movement.  Let’s again imagine that digital zombie walking down a crowded city street.  Assume we have solved the problem of making people behind him disappear behind him and people passing in front of him causing him to disappear behind them.  The zombie is staggering down the sidewalk.  Each step, his foot lands on the pavement and plants itself relatively firmly in place while he lurches forward with the other foot.

Now start to move the camera with him.  Remember, he’s being inserted into the image seen through a camera.  Normally, the zombie will move with the camera, which means it looks like his feet are sliding along the pavement.  Or floating above the pavement.

So we have a new problem.  We have to be able to track changes in the camera view.  Cameras typically pan, tilt, and zoom.  They can also roll sideways, rise higher, or drop lower.  As the camera moves, the software that is inserting the digital zombie must know how to lock that zombie’s foot onto the ground so he moves with the “real world” environment, not with the camera motion.  This means precisely tracking the changes in the viewing angle, direction, and distance of the real world environment in which the digital zombie is moving.

In addition, the AR application has to change the angle and lighting for the digital zombie, making the zombie smaller or bigger as the camera either moves closer (zooms in) or moves away (zooms out) and shifting from a front view to a back view as the camera moves from in front of the zombie to behind the zombie, all of which is known and understood principally from analyzing the surrounding real world environment, a process that, at this juncture, even very smart computers find hard to do and smartphones find incomprehensible.

And then there’s lighting: Imagine an airplane or a cloud passing over the zombie and the city street.  The sidewalk, pedestrians, and litter blowing in the breeze are all briefly in shadow.  What happens to the digitally inserted zombie?  It would look very strange indeed if the AR app didn’t appropriately change the lighting on the zombie.

Keep in mind that the human eye/brain combination is trained and practiced at doing this routinely without effort.

Back to virtual worlds for a moment: If the zombie were in a virtual world, the software simply renders both the changes in the zombie viewing angle and the changes in the surrounding environment at the same time in the same way.

An interesting side note – one advantage that computers and AR have over the real world and vision: While it will take computers (especially mobile devices) decades to match the processing power of your average dog’s visual cortex, when it comes to augmenting reality, computers have access to information that dogs, let alone humans, will probably never access directly: gps data.

Here’s a thought experiment: Blindfold a human being, load him into a car for an hour long drive down winding roads, followed by an airplane ride of another hour, and then another car ride for a few hours.  Now take off the blindfold.  Unless the person has previously visited his new location and has a visual memory of it, he will have no idea where he is.

Now rewind back to the beginning of the experiment.  Put an iPhone in the pocket of that same blindfolded person and take him through the same confusing route.  Odds are that within a few seconds of taking the iPhone out of his pocket and turning it on, the iPhone will know with an accuracy level of a few feet where in the world the traveler has arrived and can display it to him on a built in mapping service.  It can even tell which way he is pointing the device.

AR applications often take advantage of gps systems to identify where they are and what buildings, streets, and businesses are nearby.  However, gps systems are not yet precise enough to plant a digital zombie’s feet on a real world sidewalk and zoom, pan, tilt, or roll past him as he staggers and lurches towards his next victim.

Even with the help of real world gps systems, AR is obviously harder computationally than VR.  True AR properly done has to recognize objects, appropriately occlude foreground and background objects, and keep digitally inserted objects tied to changes in the environment regardless of the camera angle or movement.