Continued from #2: First build and run

It's been a couple weeks since my last post, but I haven't been idle during that time–my Cordova efforts have been channeled into an MSDN Magazine article on Cordova, and now that I have that draft done I can share more of the insights here, meaning that I can ramble on with more details than I can fit into an article! :)

First of all, I modified my markup to be a little more generic and flexible for the app. One change is to drop the map display from the app altogether in favor of a simple field for the location. I did this because the map element took up a lot of space that I didn't have on small phones, and also created more complexity for layout. In the end, the map really isn't important to the app, and by making it a text field the user can also edit the location information if they want.

Another change is to add an explicit Share button because platforms other than Windows don't have the swipe-in-from-right-edge gesture to get to it, and don't have a system-level sharing feature either. (That's something we'll deal with later on.) I could have made use a WinJS app bar for this, but I decided to keep it simple and just create a div with a button.

I also just made all the content areas <div> elements instead of using <header> and <section> and so forth. Here's the new markup:

<body>
    <div class="mainContainer">
        <div class="title mainChild">
            <h1 class="titlearea ">Here My Am!</h1>
        </div>
        <div id="locationSection" class="mainChild subsection">
            <h2 class="section-title">Here:</h2>
            <div id="locationControls">
                <input id="txtLocation" type="text" placeholder="tap to edit" />
                <button id="btnLocate" class="iconGlobe"></button>
            </div>
        </div>
        <div id="photoSection" class="mainChild subsection">
            <h2 id="photoHeading" class="section-title">My:</h2>
            <div id="photo">
                <img id="photoImg" src="#" draggable="false"
                    alt="Tap to capture image from camera" />
            </div>
        </div>
        <div id="shareSection" class="mainChild">
            <button id="btnShare" class="shareButton" disabled>Share</button> 
        </div>
    </div>
</body>

I should note that one of the complications that came up in migrating the Windows Here My Am! app is that the map I originally used was either displayed in an iframe with an ms-appx-web:// src or an x-ms-webview element, both of which are specific to Windows. I had to do this because the map area used remote code for Bing Maps. Now Cordova apps on all platform other than windows automatically run inside a webview, which means I could have just references the Bing Maps script directly, but that would (a) not work if the device is offline, and (b) wouldn't work on Windows because you can't load script into the default local context in which index.html runs. Upon further investigation, there really isn't a workaround for this at present: your root page in a Cordova app on Windows runs in the local context, but on other platforms runs in a web context. It's a difference that you just have to live with for now, and in my case I just dropped the map feature.

My next step is to style the markup it so it looks like the following:

HereMyAm_Cordova_3-1

The big trick here is to understand the impact of Cordova's nature on your HTML/CSS work. On Windows 8/8.1 and Windows Phone 8.1, a Cordova apps runs as a native JavaScript app; on all other platforms it creates a native (non-JS) wrapper around a webview. Whatever the case, the engine that renders your HTML, CSS, and JavaScript is related to the browser of the underlying platform. On Windows it's IE (10 or 11, depending on the OS); on Android it's the Android browser, (Chrome-based, of the version that matches the OS), and on iOS it's iOS Safari (7 or 8).

Which means: a Cordova's app CSS must work in each target platform's webview, just like web applications must work in different browsers by employing the appropriate prefixed and non-prefixed styles. In short, everything you know about responsive web design applies with Cordova.

As noted in the last post, I discovered this right away because the CSS grid upon which I rely heavily on Windows isn't implemented on these other platforms. And that's not all–there are many variances and differences. But thankfully there's the site caniuse.com that will tell you what parts of HTML5 and CSS are and aren't available on which platfoms, and whether or not they need prefixing. Super helpful! I suggest you visit that site and get familiar with it.

What I found was very helpful at this stage (aside from going to a CSS framework which would relieve me from the details), was to first create a Windows app project with my HTML, and then open it in Blend for Visual Studio and do my Windows-based styling. In this case I needed to use flexbox for the layout, knowing that I couldn't use the grid. Blend gave me an easy way to work out things like margins and media queries in a very friendly environment, setting the baseline styles for the app.

Along the way I also decided that given the small size of many phone screens and the fact that I eliminated the map section, it didn't make sense to support a landscape orientation. We'll return later on how to enforce that in a Cordova app; at this point it just meant that I didn't need to worry about landscape orientations at all.

With my baseline CSS from Blend, I then dropped the HTML and CSS onto my web server; you can see it all at http://www.kraigbrockschmidt.com/src/heremyam/index.html (and it has some JS in there too now). What this allowed me to do is check the styling in a variety of browsers, and then use the browser's developer tools to find the necessary styling for that browser.

On my Windows machine I opened the site in both IE and Chrome; in the latter I worked out the webkit styles for flexbox. Once I did that and updates index.css on the web host, I then opened the same page on my Android tablet to verify the layout. Then I switched to my MacBook (which I acquired to build for iOS) and checked things out in Safari, and then also opened the site on my iPad. I opened the site on my Windows Phone too. (And, I might add, that creating a short URI saves a lot of trouble entering it into a mobile browser.)

I liked using the browsers here for three reasons. First, the developer tools let you change the live CSS and see the result, just like Blend does for Windows. Second, the mobile browsers especially are very clearly linked to the webview that a Cordova app will use. Third, a browser is easily resizable, allowing you to test and style for smaller screen sizes. For example, here are the IE, Chrome, and Safari browsers sized down to a minimum, with styles for <= 499px widths and <= 480px heights, which reduce font sizes and tighten up margins:

HereMyAm_Cordova_3-2

Here's the resulting CSS. Note that use of flex: 1 1 0px; and -webkit-flex: 1 1 0px; to expand certain elements to full available space (the location input control horizontally, the image area vertically). The #btnLocation::before rule is what sets the globe icon in the locate button, and other styles on the button make it circular.

@-ms-viewport{width:auto!important}

/* 
    Notes:
    Windows and Windows Phone (through 8.1) use -ms prefixed styles.
    Android and iOS use -webkit prefixed styles.

    There are a few general design breakpoints for layout:
    Heights: 480 and 768 (actually 767, see below)
    Widths: 480, which accomodates the 500-pixel width of Windows split-screen views.
    
    If you use device-[height | width] then there are more variations. Using
    [min | max]-[width | height] you get automatic scaling for CSS, which reduces
    the breakpoints. This has the effect that some device sizes don't translate
    exactly: the iPad's 768 dimension comes through as 767.78 so the breakpoint
    has to be 767 as CSS media queries operate on integer comparisons.    	
*/

/* 
    This app will focus on being portrait-only to simplify the layout
	requirements. The only adjustments we'll make, then, are for 
	various screen sizes and not for orientation.
*/

.mainContainer {
    /* Windows */
	display: -ms-flexbox;
	-ms-flex-flow: column;	

    /* Android, iOS */
    display: -webkit-flex;
    -webkit-flex-flow: column;
	
	height: 100%;	
}

/* Override of WinJS */
h3 {
    font-weight: 300;
}

.mainChild {
    margin-left: 20px;
    margin-right: 20px;	
}

.title {
	margin-top: 10px;	
}


.subsection {
	display:-ms-flexbox;		
    -ms-flex-flow: column;	

    display: -webkit-flex;
    -webkit-flex-flow: column;		
	
	margin-top: 15px;	
}

.section-title {
	/* height: 40px; */
}

#locationControls {
    /* Use a row flexbox here to allow the input control to fill the space */
	display:-ms-flexbox;		
    -ms-flex-flow: row;	

    display: -webkit-flex;
    -webkit-flex-flow: row;		
}

#txtLocation {
	flex: 1 1 0px;  /* Expand to fill available space */    
    -webkit-box-flex: 1 1 0px;   
    -webkit-flex: 1 1 0px;
}

#btnLocate {
    margin-left: 15px;

    /* Make it circular */
	min-width: 40px;
    width: 40px;
    border-radius: 20px;
}

#btnLocate::before {
    /* WinJS references ../fonts/Symbols.ttf from its stylesheets */
    font-family: "Symbols";
    font-weight: normal;
    font-size: 10pt;    
    content: "E12B"; /* globe */  
}

#photoSection {	
	flex: 1 1 0px;        
    -webkit-flex: 1 1 0px;    
    min-height: 100px;
}

#photo {	
	border: solid 1px gray;	
	width: 100%;    
	height: calc(100% - 40px); /* 40px is height of heading */
    margin-top: 6px;
    margin-bottom: 10px;

    /* Make sure the image is centered inside this area */
    display: -webkit-box;
    display: -ms-flexbox;    
    display: flex;
    justify-content: center;
    align-items: center;

    /* iOS */
    display: -webkit-flex;
    -webkit-flex: 1 1 0px;
    -webkit-align-items: center;
    -webkit-justify-content: center;
}

#shareSection {
    display: -webkit-box;
    display: -ms-flexbox;
    display: -webkit-flex;
    display: flex;
	-ms-flex-pack: center;	
    -ms-flex-item-align: center;
    justify-content: center;
    align-items: center;
    -webkit-justify-content: center;
    -webkit-align-items: center;

	margin-bottom: 15px;
}

.shareButton {
    margin-left: 8px;
    margin-right: 8px;
}

/* Adjustments for narrower/shorter screens */
@media screen and (orientation: portrait) and (max-width: 499px),
       screen and (orientation: portrait) and (max-height: 480px)
 {	
    h2 {
        font-size: 14pt;
    }

    button, input {
        font-size: 10pt;        
    }
}


/* Tighten up margins for short screens */
@media screen and (orientation: portrait) and (max-height: 480px) {
    .title {
        margin-top: 5px;
    }

    .mainChild {
        margin-left: 10px;
        margin-right: 10px;
    }

    .subsection {
        margin-top: 10px;
    }

    button, input {
        min-height: 15px;
    }

    #photoSection {
        margin-top: 5px;
    }

    #photo {
        margin-bottom: 5px;
    }

    #shareSection {
        margin-bottom: 5px;
    }
}

 

It's important to note–and you'll see this if you run in the Ripple emulator–that the sizes reported to CSS are scaled sizes, especially where max-height, max-width, and so forth are concerned. I would suggest sticking with these instead of trying to get device pixels, because doing that would only increase the complexity of your CSS for no benefit.

 

 


Switching platform targets in Visual Studio

One of the most important things to know when working in Cordova is how to switch between target platforms. There aren't obvious controls in Visual Studio for this, but it's there. Either use the Build > Configuration Manager menu item, click the Debug/Release drop-down in the toolbar and select Configuration Manager:

cordova1-1

Once you're there, the Active Solution Platforms drop-down is where you set your target.

cordova1-2

Doing this will change the main VS toolbar to reflect options available for that target. This is how you switch between targets and run the app on devices and emulators.

Tip: to shortcut this you can add the platform drop-down directly to the toolbar. Right click the toolbar and select Customize at the bottom of the long list. In the dialog that appears, click the Commands tab, then select Toolbar and Standard:

cordova1-6

Now click Add Command, then pick Build > Solution Platforms:

cordova1-7

This adds the platform drop down to the toolbar. You might need to use the Move Down button in the Customize dialog to place it where you want. I also used the Modify Selection button to narrow the size to 130, and placing it where I wanted I see this on my toolbar:

cordova1-8

Running the app, and first issues encountered: varying CSS support

With everything migrated now, I was able to run the app on the Local Machine (Windows) as expected. The one caveat I found is that Cordova at present targets Windows 8.0 and not 8.1 (I think Cordova version 3.6 is supposed to do 8.1), which means that it doesn't understand variable-sized views, and only snapped/filled states. This means that in a narrower view that's later than 320px you get the 320px snapped view inside the wider view like you do with any Windows 8.0 app on 8.1.

On Windows Phone the app ran great in portrait mode, but the layout wasn't working in landscape. Hard to tell why, because the present Cordova targets Windows Phone 8.0 which means the app is running inside a webview inside a Silverlight app. This means there's no debugger support whatsoever, so I can't poke into the DOM to see what's going on.

On Android the layout is all wrong…because an Android webview in which a Cordova app runs doesn't support the CSS3 grid specification as we enjoy on Windows. Urk. I've relied heavily on grid layout but clearly I'll have to convert to flexbox, which is why I said CSS comes up early.

On iOS (in the Ripple emulator), looks like we have the same issue.

The upshot here is that when working in Cordova, your HTML/CSS work is going to be very much like writing responsive web pages for different browsers, because that's really what's happening with Cordova: your code is running inside a webview which has varying capabilities depending on the target platform. 

 

Finding your DOM in the Ripple Emulator

When running on my Android device, Visual Studio's DOM Explorer works just like it does for a Windows app–you can do in and tweak styles and so forth.

cordova1-3

When running in the Ripple emulator, you still get the DOM Explorer except that you see everything that's showing in the emulator, including all the surrounding controls:

cordova1-4

Egads! Where is my app? Well, the easy way to find it is through that click-to-select button in the upper left above: click that, then click in the emulator, and you'll see the DOM tree expand to where your app is. You'll see that it's located under the sections ui > middle > device-container > viewport container > and then the iframe "document":

cordova1-5

 

What's very cool in the DOM Explorer is that the CSS editor does inherently known what styles are and aren't supported for that target device. I can see right away, for instance, that display: -ms-grid; isn't among those supported for Android and iOS webviews, which isn't at all surprising. So the next job is to convert my grid-based CSS to cross-platform flexbox, which will be the next post.


I said that CSS would come in early, but the first thing to discuss in migrating the Here My Am! files from my existing Windows Store app project to a Cordova project. For this I'm just starting with the HereMyAm2a code from Chapter 2 of my free ebook. And in fact I'll keep it simple for starters and just migrate the HTML and CSS, leaving off the JS code for now.

Preliminary notes/build environment

Although I'd started working on a universal Windows app migration for this project, it's not a necessary step for working with Cordova, because it also produces cross-platform apps that will work on Windows and Windows Phone alike, as well as iOS and Android. With Cordova, you let plugins worry about platform-specific things–you don't want that kind of code in your project. 

In addition, it's helpful to know that you don't need a Mac as a build server to work with Cordova until you really want to produce an app for the store. With the Visual Studio extension installed, you can run a Cordova app directly in the Ripple emulator , which is very cool in that you can switch between devices, resolutions, and orientations (including between target platforms) much like Blend for Visual Studio does for Windows, so you can see just how (badly) your CSS is performing.

For myself, I'm working on a Surface Pro 3 as my main machine, have a Windows Phone hanging off it, have an Asus MemoPad7 as an Android dev device, which I wrote about recently. My team has a Mac laptop set up in one of my co-worker's offices as a build server (also for Xamarin) which I can get to over VPN to the Microsoft corpnet from my home office…pretty cool!

Project structure

I started with a new Cordova project (File > New > Project > JavaScript > Multi-Device Hybrid App > Blank App (Apache Cordova)); this is what you want to do rather than trying to convert an existing Windows project. I then migrated the appropriate files over:

Windows Project File Cordova Project File
default.html index.html (This is the default; you can use default.html if you change the start page in config.xml)
js/default.js scripts/index.js (didn't migrate the code yet; the project template uses scripts instead of js as a folder…I might change this later)
package.appxmanifest config.xml (the Cordova generic manifest)
css/default.css css/index.css
html/map.html html/map.html (there is no default html folder, so I created one just to hold this file for the time being)
images/taphere.scale-100.png images/taphere.png; I'll be switching to draw this with a canvas later on, so it's just a placeholder.
images/<logos> Not migrated yet as these need to be specific for each platform, whcih I haven't done yet.
WinJS references WinJS is not a cross-platform, open-source library, an instead of having it in the project references I downloaded WinJS 3.0 and brought it directly into the project:

  • /css/frameworks/ui-dark.css (you'll get a whole ton of warnings on styles in here because of the differences in CSS support between platforms)
  • /css/fonts/Symbols.ttf (ui-dark.css and ui-light.css assume this file is in ../fonts, so it needs to be in a sibling folder unless you modify the reference)
  • /scripts/frameworks/base.js
  • /scripts/frameworks/ui.js
  • /scripts/frameworks/WinJS.js

I don't know how much of WinJS I'll be using, but given that I have lots of experience in this area, it seemed like a good opportunity to keep up! Plus the ui-dark.css has styling that I'm used to with this app. We'll see how it goes.

  The project template also includes a merges folder for platform-specific code, and a res folder for resources that I'll get to later on.

 

Updating index.html

With this migrated project, I fixed up the references in index.html to bring in the appropriate files:

    <link href="css/frameworks/ui-dark.css" rel="stylesheet" />    
    <link href="css/index.css" rel="stylesheet" />    

    <!-- Cordova standard -->
    <script src="cordova.js"></script>

    <!-- WinJS -->
    <script src="scripts/frameworks/base.js"></script>
    <script src="scripts/frameworks/ui.js"></script>
    <script src="scripts/frameworks/winjs.js"></script>

    <!-- Project references -->
    <script src="scripts/platformOverrides.js"></script>
    <script src="scripts/index.js"></script>

Note: in a Windows project, especially with WinJS page controls, it's usually helpful to prefix all references with a leading /. You do not want to do this with Cordova because it doesn't work. I'll have to see what happens later on when I try using page controls.

Note also that you need the reference to cordova.js even though it doesn't exist in your project.

With my script/css references fixed up, I just use the same markup as before in index.html:

<body>
    <div id="mainContent">
        <header id="header" aria-label="Header content" role="banner">
            <h1 class="titlearea "> 
                <span class="pagetitle">Here My Am!</span>
            </h1>
        </header>
        <section id="section" aria-label="Main content" role="main">
            <div id="photoSection" class="subsection" aria-label="Photo section">
                <h2 class="group-title" role="heading">Photo</h2>
                <img id="photo" class="graphic" src="images/taphere.png" alt="Tap to capture image from camera" role="img" />
            </div>
            <div id="locationSection" class="subsection" aria-label="Location section">
                <h2 class="group-title" role="heading">Location</h2>
                <div id="map" class="graphic" aria-label="Map"></div>
            </div>
        </section>
    </div>
</body>

 

Up next, running the app.


With my new role as a Content Developer in Microsoft's Developer Division, I'm working now with cross-platform tools like Apache Cordova and Xamarin, and dabbling a little in Unity, and also looking at the Visual Studio ALM tools and how they apply to these project types.

To help myself ramp up on Cordova, I'm in the process of migrating Here My Am!, the app that I build throughout the pages of Programming Windows Store Apps with HTML, CSS, and JavaScript, 2nd Edition, to Cordova. I can already see there will be numerous challenges to overcome, which I'll try to write about as I work through them (preview: the first post or two will be about CSS). 

For all this I'm using the Multi-Device Hybrid Apps Extension (Preview) for Visual Studio 2013 Update 3 (how's that for a name!). One thing that's immediately clear about the extension is that it saves a whole lot of trouble in getting a Cordova environment set up. Normally you have to go get a bunch of downloads from nine different places, some of which have installers and some of which are just ZIP files, and then you have to install them in the right order figure out how to configure each one with the right environment variables. The extension does that for you.

The documentation for the extension is found on http://msdn.microsoft.com/en-us/library/dn771545.aspx; the installer page specifically describes what pieces it downloads and how to do the configuration, manually. My team is, in fact, the one that produces this documentation–more to come!


In a number of forum questions over the years, I've seen devs struggling with using relative URIs to in-package resources. Typically it boisl down to the difference between something like "images/pix1.jpg" and "/images/pix1.jpg".

These two URIs mean, in the words of Jeremy Foster, "dive into the images folder and find pix1.jpg" and "go back to the root, then dive into the images folder and find pix1.jpg", respectively.

In Windows Store apps, it's helpful to remember that all such in-package 'relative' URIs are shorthands for ms-appx://<package_id>, as in ms-appx://<package_id>/images/pix1.jpg or just ms-appx:///images/pix1.jpg. A leading / on URIs, then, is what really makes the reference an absolute one, rather than a relative one. It just feels relative because you're not specifying a schema.

To be even more precise, a relative URI always must resolve against some base URI, which is defined by the referring document. Where people get tripped up is that when you're using WinJS page controls, such as ms-appx:///pages/somePage.html, then ms-appx:///pages becomes the base URI, and images/pix1.jpg will resolve to ms-appx:///pages/images/pix1.jpg. If you thought you were referring to the root images folder in the project, then you'll likely find the image not appearing at all. If you use /images/pix1.jpg, on the other hand, then you are making an absolute reference from the package root via shorthand, which resolves to ms-appx:///images/pix1.jpg.

The bottom line is that unless you really know that you're making a relative reference to the currently loaded HTML page, use a leading / to get back to your package root.


When you have a Webview hosting an iframe element, it's possible to monitor what's going on in the iframe using the MSWebViewFrame* events, which are documented with the x-ms-webview element. These are MSWebViewFrameNavigationStarting, MSWebViewFrameContentLoading, MSWebViewFrameDOMContentLoaded, and MSWebViewFrameNavigationComplete.

In the event you have multiple iframe elements in a webview, the way to distinguish between frames is with the e.uri property, where e is the event args object for the event in question. This property is presently undocumented, but should be–this comes directly from the engineers who built the webview, so we can trust that it's there!

Generally speaking, the content of a Webview is generally a black box from the host app's perspective, and for this reason you can't get into the Webview's DOM, you can't get element references (e.g. to an iframe), and so on. There'd be serious overhead to go that route.

Which brings us to the question of what happens when code within an iframe or Webview crashes. You've certainly seen some of those sites–the ones that cause the browser to pop up debugging messages about JavaScript exceptions and whatnot. Overall, crashes in a Webview or iframe should not crash the host app.

However, there is the possibility that long-running scripts within whatever web page you're hosting are taking so long that your app is considered to be unresponsive. Those are messages you've seen in the browser as well, and if the app gets blocked on the same kind of script, it's possible the Windows will take the app down. This is a known issue that might be addressed in future releases of the operating system. At present, you can consider watching the Webview/iframe NavigationStarting and setting up your own timeout so you can catch an unresponsive site before Windows takes down the app.


I've seen the question come up on occasion about using WinJS within a webview. This is entirely supported, and has been since the introduction of WinJS–the library always checks whether WinRT is present and adjusts itself accordingly. For example, if WinRT is available, then it will use WinRT app data for saving its settings; if WinRT isn't available, it uses standard HTML5 APIs instead.

This strategy wisely foresaw the day when WinJS was licensed for use in web page. This wasn't originally the case (as the copyright headers in the WinJS files made clear), but with WinJS going open source in April 2014, WInJS can be used anywhere.

When using WinJS, and especially WinJS controls, inside a webview or web page, it's important to remember steps that are included by default within the Windows app templates, most notable the step of calling WinJS.UI.processAll. That is, when you create a new Windows app, the templates in Visual Studio supply the following default activation code:

var app = WinJS.Application;
var activation = Windows.ApplicationModel.Activation;

app.onactivated = function (args) {
    if (args.detail.kind === activation.ActivationKind.launch) {
        if (args.detail.previousExecutionState !== activation.ApplicationExecutionState.terminated) {
            // TODO: This application has been newly launched. Initialize
            // your application here.
        } else {
            // TODO: This application has been reactivated from suspension.
            // Restore application state here.
        }
        args.setPromise(WinJS.UI.processAll());
    }
};

Clearly, if you're using WinJS outside of an app, then the Windows.ApplicationModel.Activation namespace won't mean anything, so naturally you'll delete all the code inside the onactivated handler here. But if you delete WinJS.UI.processAll then none of your WinJS controls will show up. This can be confusing and might lead to the idea that WinJS doesn't work in a webview. But it does, because in the end WinJS is just a JavaScript library using standard HTML5/DOM to do its work. So just make sure that somewhere you have a line like this:

<script>
    WinJS.UI.processAll();
</script>

 


With the introduction of universal Windows apps at //build 2014, and the announcement of Windows Phone 8.1 that includes support for writing apps in HTML, CSS, and JavaScript, we do have some resources that are starting to emerge.

First, my newly-released second edition, http://aka.ms/BrockschmidtBook2, is very applicable to Phone apps. I didn't have time to spell out all the differences, but there is a summary of what to watch out for in Chapter 1, namely a few differences in controls and some parts of WinRT that aren't on the Phone.

Second, watch the Building Apps for Windows Blog for material in this area. For example, an upcoming post will spell out the controls story between Windows (WinJS 2.0) and Windows Phone (WinJS 2.1).

Third, Josh Williams and Ryan Salva did a demo session on building a universal app with HTML, CSS, and JavaScript at //build, which you can find on http://channel9.msdn.com/Events/Build/2014/2-540. What you'll see is that they do nearly all of the work in the Shared folder of the VS project, making only one small change at the end of the session to switch the Hub control in the Windows Store app to Pivot in the Phone app…the two controls, in fact, are identical in their API except for the top-level object names, so you just change the names and voila! You're in business.

Finally, there is a post on the Visual Studio blog on universal apps: http://blogs.msdn.com/b/visualstudio/archive/2014/04/08/building-windows-phone-8-1-apps-in-html.aspx.

More is certainly to come.


As of last week I assumed management and content development for the Windows App Builder’s Blog, which I’m sure most of you follow already.

What I’d love to hear from you is this: what content would you like to see on the blog? What questions are you dying to have answered? What ‘inside stories’ from the Windows engineering team would you like to know? Let me know in the comments!


A while back I worked with a forum question that asked whether the canvas element’s ImageData property for Windows Store apps had this interface:

interface ImageData {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
readonly attribute CanvasPixelArray data;
}

Or this one:

interface ImageData {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
readonly attribute Uint8ClampedArray data;
}

Where ImageData comes from:

var imageData = ctx.getImageData(0, 0, canvasWidth, canvasHeight);

This was in regards to some pixel manipulation methods using the Uint8ClampedArray.

The answer is that for Windows 8/8.1 and current IE implementations, the canvas provides a CanvasPixelArray. To be honest, the performance difference between the two matters most if you’re just measuring how quickly you can access the pixels. In practice, if you’re doing any meaningful manipulation on each pixel, that computation is probably much more significant.

For pixel manipulation, it does make sense to get the pixels into a WinRT component written in C++. Here, it is possible to pass a CanvasPixelArray to a WinRT component that accepts a byte array, wherein the whole thing will be faster than using a clamped array to begin with. Here’s an example:

void Class1::CreatePattern(WriteOnlyArray<uint8>^ data, int canvasWidth, int canvasHeight)
{
uint32* data32 = (uint32*)data->Data;
for (int y = 0; y < canvasHeight; ++y)
{
for (int x = 0; x < canvasWidth; ++x)
{
uint8 value = x * y & 0xff;
data32[y * canvasWidth + x] =
(255   << 24) |    // alpha
(value << 16) |    // blue
(value <<  8) |    // green
value;             // red
}
}
}