wbyoung

Whitney Young is a developer at FadingRed
 

When Mac OS X 10.7 and iOS 5 were released, I was so excited about the new features in Core Data. The additions of explicit concurrency types, nested contexts, and iCloud syncing promised to make developing with Core Data even more delightful. After evaluating these features, our team decided that we were going to use them in the next major release of one of our apps. Unfortunately (as is sometimes the case), some of these features aren’t quite ready for prime time.

Today I’m going to discuss nested context support in Core Data and various issues that exist with them. Some of these issues are minor bugs. Others may be considered functionally correct by the Core Data team, but result in unexpected behavior. Regardless, they add up to nested contexts being a feature you should avoid completely (as of this writing).

Background

So what are nested contexts anyway? To avoid a very lengthy explanation, I’m going to start by assuming that you understand the Core Data stack. Nested contexts allow you to set up a managed object context so that it accesses data from a parent context instead of from a persistent store. If you request an object from a managed object context that has a parent context, Core Data will first look in the parent. If the parent context has that object in memory, you’ll get a new managed object just like that one. So if there are changes in the parent, you’ll get the changed version of the object. If the object doesn’t exist in that context, it will keep going up through parent contexts until it finally fetches the data from the persistent store. Basically, retrieving data goes all the way up through all ancestor contexts. Saving, on the other hand, doesn’t traverse up through parent contexts — it only saves one level up. All of this is pretty well explained elsewhere, so I’m going to leave it at that.

Issues

Throughout the last 9 months, we’ve been using nested contexts in an attempt to simplify and improve some of the data processing that we do in our personal finance application, Koku. It feels like at every turn we’ve been finding issues in Core Data where nested contexts are to blame. Many of these issues are a bit difficult to explain because there is a lot of setup required to reproduce them. With that being said, here is what we’ve found so far:

Child contexts block ancestors

This is one of the most fundamental problems with nested contexts. When I was first told about this through some friends in Chicago, I didn’t believe it. Basically, when you execute a fetch request in a child context, it blocks the parent context while performing the fetch. What does this mean? Well it means that you can’t really use nested contexts to solve a lot of concurrency issues.

Here’s an example: you have a bunch of objects in your database that need to be fetched and have time-intensive code executed on them. The following code looks like it would solve your problems:

NSManagedObjectContext *asyncContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[asyncContext setParentContext:mainContext];
[asyncContext performBlock:^{
    // create fetch request, execute, and do work
}];

But executing that fetch request blocks the main context. Assuming that main context is running on the main thread, then this also makes your application stop responding.

I’ve commonly heard people using this setup for importing from some type of service. And this is where things start to get bad. Let’s say you’re importing a bunch of JSON objects and you need to check if each already exists in Core Data based on an identifier. You write code that loops and checks for an object in Core Data with that identifier, then creates a new object from the JSON data if it doesn’t exist (yes, that could be optimized). It may look like everything is going fine because your UI is still responsive through most of that import (especially if you’re reading from a stream since there will be a delay between each ID check). The truth is, though, that every time you check for an object with an identifier, you’re blocking your UI. It may only be for a fraction of a second, but no one is expecting this.

Now this in itself is not really a large problem. Once you understand it, it’s just another part of the framework that you work with/around. The real problem is that nothing in the documentation indicates that this will happen. In fact, the release notes read:

This pattern has a number of usage scenarios, including:
  • Performing background operations on a second thread or queue.
  • Managing discardable edits, such as in an inspector window or view.

Right there in the release notes, Apple is telling us to use nested contexts to do work in the background. But you absolutely should not use them for that. They will block the parent.

radar://10581934

Making changes into a parent context can be slow

Sometimes the cost of making changes in a child context is outrageous. Simply changing a boolean value on a bunch of objects can be prohibitively slow. It seems that Core Data tries to do something while processing pending changes on the child context. A reasonable assumption is that Core Data is trying to negotiate changes between the two contexts. All we’ve been able to determine is that there’s a lot going on with dictionary lookup that can hang the app. Everything takes far longer than if you were doing the same thing in a context that was tied directly to a persistent store coordinator.

Fetch results are wrong in a child context

It’s possible that fetch requests will not return the right results. I cannot even begin to understand how nested contexts got released when something this fundamental is broken. (Ok, so it’s broken in a very specific way, but still.)

Here’s how you can get Core Data to return the wrong results: set up a nested context relationship just like they describe for UIManagedDocument. For those who are unfamiliar, this involves an asynchronous context that is connected to the persistent store coordinator. The main queue context sets its parent to be this asynchronous context.

With the contexts set up, create a new object. Let’s imagine this object represents a blog and its variable name is awesomeBlog.  Add a couple of articles to that blog. Now if you were to set up a fetch request for all of the articles with a predicate of [NSPredicate predicateWithFormat:@"blog = %@", awesomeBlog], you’d get back the right results. But if you save the main context and then the async context (so the data is actually persisted to the disk) and try executing that fetch request, you would get no results. And it doesn’t matter if you set up the fetch request before or after you persist the data. You’ll never get the right results. Without nested contexts, this works just fine!

There is a workaround for this issue. You can simply obtain a permanent ID for the object when you create it. But there is a catch. The async context (at least in the case of UIManagedDocument) exists so that you can write data asynchronously. By getting a permanent ID for each object you create, you’ve made it so that you’re doing synchronous writes more frequently.

Oh, and as an added bonus, everyone who uses UIManagedDocument will experience this bug.

This has been reported as fixed for an upcoming release of iOS (which means the same should be true for Mac OS X).

radar://11891033 fixed in an upcoming release

Cannot access relationship in a child context for unsaved objects

This is pretty similar to the issue above in that it deals with relationships of objects that have been created. The difference is in the context setup and that it doesn’t really involve saving.

Here’s what can happen: let’s say you’re working in a managed object context called mainContext. It doesn’t need to have a parent context; it can be hooked up directly to the persistent store. Insert a new object into that context. Let’s call it a blog again. Add a few articles to that blog. Now, create a new managed object context and set it’s parent to mainContext. We’ll call this new context editing context. It’s another one of those common workflows that Apple documented in the release notes for nested contexts — discardable edits. But here’s what happens: when you access your blog in editingContext, it has no articles.

The workaround for this is the same as the last one. Just obtain the permanent ID of the object beforehand. This has also been reported as fixed for Mac OS X 10.8 Mountain Lion (which means the same should be true for iOS 6).

radar://10209854 fixed in an upcoming release

Sorting is not honored when there are changes in a parent context

Sort descriptors are important, right? Well they don’t seem to always matter when you’re using nested contexts. If you have changes in a parent context and you execute a fetch request in the child, sorting does not work. Mix that with some of the above issues and you may get out of order, incomplete results. Yikes!

NSFetchedResultsController deadlocks

You never want your application to deadlock. With NSFetchedResultsController and nested contexts, it’s pretty easy to do. Using the same UIManagedDocument setup described above, executing fetch requests in the private queue context while using NSFetchedResultsController with the main queue context will likely deadlock. If you start both at about the same time it happens with almost 100% consistency. NSFetchedResultsController is probably acquiring a lock that it shouldn’t be. This has been reported as fixed for an upcoming release of iOS.

radar://11861499 fixed in an upcoming release

Crash after obtaining permanent IDs

A few of the issues above could be worked around by getting the permanent ID of an object. This can actually lead to a crash in certain circumstances, though. Again using the UIManagedDocument setup, create a new main queue context, editingContext, off of the existing main queue context, mainContext. Create a new object in editingContext. Save editingContext. Obtain a permanent ID for the same object in the mainContext. Core Data will eventually crash when cleaning up resources no longer in use.

radar://10480982 fixed in an upcoming release

Wrapping Up

If you made it through all of those, you’ve probably come to the conclusion that nested contexts are currently something to avoid. If not, the bullet points alone may have been enough to cause some unease. And these are only the issues that we’ve experienced. I wouldn’t be surprised if there are more. In discussions with peers, I’ve found that there are many people who are discovering some of these same issues and finding ways to work around them individually. I hope this list helps you make an informed decision about using nested contexts before you go down the same road we did. Also, please consider filing duplicate radars for those that are listed above. If you have anything to add to this list, please reach out to me on twitter or send me an email.

Thinking back over the lifetime of Core Data, it’s pretty unfortunate to see so many problems with the recently introduced features. I remember back when Core Data was released that SenTestingKit was just making its way into Xcode. At some point, Apple used Core Data as a poster child for unit testing. They claimed they were able to tune and enhance the performance in Core Data, making it screaming fast because of the comprehensive tests that they wrote. Whether this was marketing or not, the framework has been pretty solid throughout the years. The issues that have cropped up with nested contexts (and don’t even get me started on iCloud) feel like huge oversights and problems that fall nicely into the comprehensive test coverage category.

With all that being said, nested contexts are such a wonderful idea that could simplify architecture issues that so many developers using Core Data face. I look forward to the day that they work flawlessly and we get to start using them to make the next generation of data backed applications.

Update 2012.08.08: Updated information on radars. Update 2012.09.16: Updated information on radars.

August 17th, 2011 1 note

As a software developer, I’m constantly hearing people talk about California. Silicon Valley and San Francisco are the places to be. And increasingly people are talking about New York as a tech city as well. These are wonderful cities for technology. There are tons of startups, established companies, venture capitalists, like minded people, and constant tech events. Add that to the beautiful weather in California or the massiveness of New York, and what’s not to love?

A few years ago when we started FadingRed, Brittany and I were living in Chicago. Neither of us had anything that was really keeping us rooted in Chicago, so we discussed moving to San Francisco and starting our company there.

At the time, I’d never been to California, but I’d always heard good things. Throughout college, people talked about moving out west like it was the promised land. No more cold weather, beaches, job opportunities, relaxation, etc. It seemed like there was a never ending list of reasons why California was the place to be. So when we talked about starting our company there, it was a really exciting possibility. Ultimately we chose to stay in Chicago, though, and I wouldn’t go back on that decision.

Chicago is not a city that people think of when they think tech, but that doesn’t mean that there’s no tech here — there certainly is! There are developer meetups for people in various tech scenes, great conferences are held here, we’ve even got some big named companies here as well. And as for the weather, well I enjoy the seasons — we go from grilling to ice skating, and the winter really isn’t that bad. So what makes Chicago better than San Francisco? Technology isn’t all around you all the time.

I’ve been out to San Francisco a few times now, and it doesn’t take long for the pervasive presence of technology to wear on me. You just can’t escape it. In coffee shops people are talking about their next product idea. On the way to lunch, you hear people discussing nitty gritty implementation problems. You could, of course, explore the cultural side of the city and enjoy restaurants, museums, music, etc, but you’re still pretty likely to happen upon a bit of technology during your journey.

When I leave the office, I want to leave work behind. Now, it’s not that I don’t enjoy what I do. I do. But I also enjoy so many other parts of life. I want to have a break from tech so that it’s refreshing to come back to it the next day. Even when I hang out with coworkers or other people who write software, it’s nice to take a break and talk about life rather than just what’s new in the tech world. Here in Chicago, I feel like I can have that break when I need it. When I grab a cup of tea, I can sit and just enjoy the peaceful murmur of the conversations around me without my brain firing on all cylinders. And as mentioned previously, when I’m in the mood for a good tech meetup, we have those too. There’s really nothing not to love about this city. It’s the perfect place for me at this point in my life.

In the end, this is not a why my city is better than your city article. The takeaway is this: consider your city or town as a good alternative to one of the big tech cities, because it is. There’s no reason you can’t have the best of both worlds, and chances are it’s already in your backyard. Go enjoy!

March 10th, 2011 7 notes

I’ve had this uneasy feeling recently. I keep thinking that Apple is on the verge of trending toward unfavorable. Don’t get me wrong, I still love the Apple products that I use. I still think they’re a great company. But with the decisions Apple has been making recently, they’re finding themselves on the edge of a very slippery slope.

Members of the Apple community have been very vocal recently in regards to:

  • App store rejections & policies
  • Keeping 30% (including subscriptions)
  • Charging for Xcode and FaceTime

Before diving into what’s been going on recently, let’s take a quick look back…

Historically, Apple has done a great job of hacking the press and keeping an overall positive vibe for their brands. In the last few years, they’ve done a really great job. Think back to the release of the iPod. Apple was untouchable, everything they did was fantastic. Iteration after iteration, they just kept cranking out great new devices that consumers absolutely loved. They completely changed the music industry. Then they did it again when they created the iTunes store. Buying any song for 99¢ was a revelation. It came with the hitch of DRM, but that was easily pawned off as a requirement set forth by the music industry.

When we fast forward to today, there’s a lot of hitches. These hitches are not externally imposed either. Apple’s consciously making decisions that are controversial. They’ve decided that all applications running on the iPhone must adhere to certain requirements. They’ve decided that they’re going to keep 30% of sales that pass through their store. They’ve decided to charge for software that used to be free (albeit for rather insignificant amounts). When taken in isolation, these decisions are justifiable. They even make sense. And the fact that they’re controversial doesn’t matter because they’re on the backs of absolutely fantastic products. The iPhone revolutionized the smart phone industry. The app store has been a wild success. Apple keeps producing awesome products, and the controversy takes a back seat.

While Apple may still be a press darling, things are starting to change. Recently people have described Apple as too powerful, greedy even. This is not the Apple we used to know. We used to know an Apple that was one hundred percent about the consumer. We used to praise their applications for their consumer oriented design. We used to love every aspect of every product. Now we’re starting to see a different Apple.

I think what we’re all feeling is kind of like falling out of love. There a lot of little things that are adding up. Each little thing taken on it’s own is really not a big deal. It may bother you, but you can live with it. When you step back and look at the big picture, though, you start to realize that things aren’t quite the way you expect them to be.

There are a lot of people in the Apple community who I think share the same sentiments. People’s reactions to changes range from indifferent to straight up pissed. In the end, though, it’s not about price; it’s not about rules & policies. It’s about how Apple’s been acting doesn’t feel right. It doesn’t feel like they’re they Apple that we’ve all come to know and love. I’m hopeful that in the coming years we will fall in love all over again.

November 22nd, 2010 1 note

From the start you should know — are you creating a company with a team or do you plan on keeping it a one man shop? This is a very personal decision but if you don’t decide up front, adding people to your team will be significantly harder. You’ll have to determine how your team will communicate, how to handle payroll and taxes, how to hire talented people, etc. If you start on your own and don’t plan for these changes, you may catch yourself in a tough spot. Adding people is a big task, and you’ll have to tackle it while keeping the company going. Your day to day flow will change significantly. You need to be prepared for this change, and knowing from day one will give you ample time to prepare.

There are advantages either way, and there’s no wrong decision. When FadingRed started, I was the only one working full time. This lasted for approximately 6 months, then Brittany came on full time. We worked from home for another 12 months before Sara came on and we moved into a dedicated office. We’ve just finished recruiting, and have two more developers who will join us next summer. The last two years have given me a great perspective on what it’s like working in different environments, and I’d like to share a little about each.

Working Solo

The best thing about working on your own is that you can really choose your own schedule. If you want to work from 3 in the afternoon until midnight, there’s no one stoping you. If you’re really working in solitude (no kids around, no pets, etc) you’ll be able to work without interruption as an added bonus. All of this is great, but working whenever you want can easily land you in a workaholic or slacker rut. You’ll want to avoid this type of rut because no matter who you are, you need to have a balance between work and life. You need balance for your health. When your schedule is constantly in flux, you’re throwing off this balance. This may not be an issue if you’re able to define boundaries for yourself.

For me, defining my own schedule was tough. I started working almost immediately after waking up. I’m not necessarily a morning person, but getting email handled right away felt good. What this translated to, though, was that I didn’t really give myself a chance to relax and enjoy a little bit of the day before starting work. On most days, I would stop after I got a good amount done, but sometimes I would find myself working a little past dinner to solve the last problem of the day. Days when I worked longer than I should have affected the next day. The workaholic in me brought out the slacker in me. I’d spend a bit of time the following day catching up on a TV show while working. I’m not nearly as productive while multitasking in this way. This all can become somewhat cyclical.

Now none of this was really bad. I still loved every day of work, but I think that if I had continued working solo that I would have needed to better define a schedule for myself.

At Home

Working from home is great. I had the luxury of working from home when it was just me, as well as when Brittany started working full time for FadingRed. Having zero commute time is wonderful.

I don’t have much to say about working from home that hasn’t already been said. What I do think is important is that you have an office in your home. The reason being that work can feel like it’s taking over your life if you aren’t able to close the door on it when it’s over.

Small Team

A lot changed for me when I was no longer the only one in the room each day. The first thing was that my schedule became more well defined. It was still flexible, but there was definitely more structure. This change was very good for me. I felt like my productivity and focus increased and I had a much better balance between work and life. That last sentence is really important — both my productivity and focus increased. I don’t really know what caused it, but it is definitely related to working with others around you. Maybe subconsciously people feel that they need to be productive if everyone around them is productive. Regardless, this is a very positive side effect of working with others.

The biggest change you’ll experience when switching from working on your own to working with even just one more person is the addition of communication. Communication is great if used effectively, but it’s a burden if done poorly.

Good communication leads to new ideas and improvements to ideas that you really just can’t come up with when it’s one person banging his head against the wall. One way to achieve this is through brainstorming. There’s really a lot to be said for bouncing ideas off of other people before you go ahead with them. You waste less time on bad ideas.

On the other hand, communication can get in the way. Structured communication (aka meetings) are monotonous. The way to avoid this is to just allow communication to be unstructured, but what you’re left with is interruption. Interruption is the arch nemesis of productivity. When one person interrupts another, it’s great for the interrupter; they get the information they need right away. The person who is being interrupted, though, has to switch her focus to address the other person, concentrate to respond meaningfully, and then re-focus on her task. This can be very frustrating and can take a lot out of you mentally. Managing communication effectively is something that many teams struggle with, and it’s because there isn’t a singular solution. The solution will be different for every team and even for different members within the team. Just know this ‚Äî no matter how well you know your co-workers, you will always be working to improve communication.

Office Life

These days I wake up and leave home to go into the office. FadingRed has a small office in downtown Chicago. After working at home for nearly a year and a half, an office is an adjustment. I know a lot of people who work from home and love it. In conversations I’ve had recently, these people kind of wrinkle their faces at the idea of switching back to office life. For me, it’s been great.

My commute is short. I live downtown and walk to work. I no longer have a home office. My work is totally separate. Don’t get me wrong, I love what I do, but working all day every day just leads to burn out. Now it’s easier to focus on other interests in the evenings and on the weekends. Work isn’t there peeking out at me from the home office. I still work from home occasionally, so it’s really the best of both worlds.

For me, this is what I imagined from the time that I started. I never thought that working by myself cooped up at home all day would work long term. For others, that might be the right choice.

Plan It Out

These have been my experiences. Whatever you’re thinking, set yourself on the right course on day one. Think about where you see yourself and your company in the future. You don’t need to have a 5 year plan (or even any plan at all). Just an idea. Even small decisions can have a lasting impact. If you think you’ll have a team, imagine them there with you at the beginning. The transition will be easier if you don’t have to change everything all at once.

September 16th, 2010 2 notes

Find

You’ll find it randomly. There’s no telling how your perfect job will materialize. You might find it from a job posting. Maybe it was a website that you ended up browsing some day. How you found it doesn’t matter, but when you do, your thoughts will be:

This is where I want to work.
It’s my dream job.
I have to get it.
I need this job.
I’ll do whatever it takes.

Apply

What you need to do is simple — apply to work with people. A company is nothing more than the people who work there. Learn what you can about them. Online social networks have made it incredibly easy to learn about people. Find out what makes them tick; you’ll probably find something that makes the company even more intriguing. Absorb this information.

Before you apply to the company, really think about everything you’ve learned. For a long time. This is your dream job, remember? Talk to your friends and family about it. Listen to what you’re saying because it’s these thoughts that you need to express in your application. Talking about it should be something that just happens because you’re excited — if not, it’s probably not the right job for you.

Now you’re ready to apply. There’s only one thing you need to do; you need to convince someone who you’ve never met before that you’re telling the truth. This is the hardest part. The rest of your application is easy. Skills, experience, whether you’re a hard worker — these don’t matter. If you’re applying for this job, it’s because you know you’re capable of performing the work. Put your energy into telling your story.

This story is all about you and your relationship with the company. You’re in love with the company, and you have to convince the people who work there to love you back. How did you find out about the position? Why is it different from what you’ve done in the past? What makes it perfect for you? If you really want the job, don’t re-use something you wrote before; start from scratch. Make it good, then proofread it seven times. You can tell your story in an email or a more formal cover letter — it doesn’t matter — the content matters. Think about it from the perspective of someone who doesn’t know you. Does it convince them you’re perfect for the job? Is it genuine? Make sure it says what it needs to and nothing more. Keep it focused.

We have a small team. Right now it’s three people. When you send in an application it goes to all of us. We all look at it. There’s no human resources barrier to get past. Your story will be read. Make it count.

Face to Face

Once you get an interview, it’s time to start over. Repeat your research. You want everything you know about the company in the front of your mind when you start meeting people, and there may be new information to learn. Every second your with these people will be time that you’re evaluated. If it’s a small company like ours, make sure you know everyone’s names.

Once you’re in the door, put some of your research to work. “I read on your blog that you want to build a company where all customers are given first-class treatment, and I really respect that.” Share why this is important to you. This is relevant and shows that you’ve spent your time researching the company. Just be sure that you don’t end up being creepy. “I read on your Facebook profile that you enjoy playing tennis and I think it’d be cool to play tennis with my co-workers.” This has nothing to do with the job. It may also feel a little invasive.

The actual interview is always difficult. You will feel nervous. You will stumble at some point, but it’s okay. Your interviewer will be forgiving. Interviews are high-stress situations, and no one is able to perform at one hundred percent. Let it pass. Clear your mind and move on. Remember that interviews are a two way street. You are interviewing the company; make sure you have good questions to ask. Even if you think you know the answer, ask anyway. You need to get your interviewer engaged. You need conversation. At the end of the day, your interviewer needs to know that she enjoys conversing with you.

Follow-up. There’s no better way to screw up an interview than to fail to follow-up with your interviewer. If there’s more than one person you interviewed with, follow-up with all of them. In order to do this, you need to make sure you get their business card. If all you get is a business card from someone in HR, email the HR person to say thanks, and ask for the email addresses of the people with whom you interviewed. This is a lot of work. Do it. Remember, you’re applying to work with people. People appreciate and expect your etiquette.

Rinse and Repeat

If you’re lucky, this section will not apply to you. For many people, though, a career has become a series of job changes. You’ll repeat the whole process a number of times. Those of you who are lucky enough to have found a job and a company you love, cherish it.

Best of luck!