Where is Windmill on the iPhone?

Approximately 10’ minutes reading time.

Moving the needle

I am hearing from the developer community there is a broader concern around the restrictions and the limitation on the tools Apple provides and allows. I want to kick-off a discussion about this to address these shortcomings. How should Apple enable developers to improve on the tooling for the iOS development ecosystem? I’d love to hear your thoughts.

Some of the questions to get the conversation going:

  • How are existing developer tools limited when building for Apple Platforms?
  • Are you using a 3rd party developer tool and why?
  • Did you feel the need to build a developer tool, why?
  • How should Apple improve on its tooling and resources made available to developers?
  • Do you think iOS should have a dedicated “Developer Tools” section?

I will add what you’re saying below. Please mention @windmill_io to help me track your responses.

I wish Apple were better at collaborating with the developer community, instead of just controlling it. If Apple stopped trying to do everything itself, focused on just building a great platform, and let us solve our own problems, everyone would win.

Apple should be in the business of fostering creative solutions to lots of different problems instead of shutting them out.

Let’s see if planting the seed of making the language and some frameworks open source can grow into a more open ecosystem of tooling. The closeness of Xcode is one of the things that annoys me the most.

I don’t understand all the issues here, I admit, but I start by thinking that useful developer tools should be allowed on the App Store.

Apple’s loss ultimately when people stop innovating for their platforms

Table of Contents

Introduction

Windmill on the iPhone allows an organisation with an iPhone app under development to make that app available for distribution on any devices they own, as documented in the iOS Deployment Reference.

It makes use of ad-hoc distribution to make it dead easy to get started as a small- to medium-size business, with app distribution to half a dozen or so devices, while progressing towards TestFlight.

It allows businesses to distribute their app continuously while still in development, at regular time intervals, on demand, or as a reaction.

  • As individual contractors, it enables them to show progress to a client offsite and deploy bug fixes, at will, in time-critical moments.
  • As part of a team, it enables them to have nightly builds to get daily feedback from a product owner.
  • As part of an organisation, it enables them to have weekly meetings with the business and marketing team to do a demo, discuss, and plan next week’s development.

Windmill on the Mac continuously monitors the codebase, as made available in a git repository, so that every time a developer pushes a code change, it ensures the app still works. A developer starts using it as soon as they start working on an iPhone app.

It runs on Apple hardware that the organisation owns, on their premises. Therefore, it does not expect them to hand over their Apple Credentials, Distribution Certificate, Private Key, and Provisioning Profile, making it more secure than systems asking developers to hand these files over and keep them stored on their behalf.

Windmill can scale up to a platform where an organisation runs multiple instances of Windmill on the Mac on the available hardware to define a workflow and enforce security requirements.

An organisation in a small office with 2 developers and 1 product owner would have each developer running Windmill on their development Mac monitoring their respective feature branch, as well as an instance of Windmill running on a dedicated Mac mini monitoring the master branch.

This way, each developer gets feedback on their own codebase while the Mac mini holds the Distribution Certificate and Private Key to distribute releases and report on merges. This allows an organisation to have an overall view, a shared knowledge and centralised security on their app. Does the app compile? Are tests available and passing? Can the app be installed on a device? Can we release on the App Store?

Rejected

Unfortunately, Apple has firmly rejected Windmill on the iPhone. Windmill on the Mac does not seem to have Apple’s blessing either.

To Apple’s credit, they did take their time evaluating Windmill on the iPhone and Windmill overall.

Windmill on the iPhone was in Review for over 20 work days and an additional 5 after I sent an eight-page “Letter of Appeal” to the App Review Team with the following:

  • An overarching description that explains what Windmill does, why I think it’s useful for developers, why it does not violate the App Store Guidelines and Apple Developer Program License Agreement, and how Apple benefits by approving Windmill.
  • The security considerations and implementation details that went into Windmill, for full transparency in an effort to make Apple feel comfortable about Windmill.
  • In detail, why I believe Windmill complies with the App Store Review Guidelines and the Apple Developer Program License Agreement.
  • Some thoughts on how the Mac and the iPhone are governed by two very distinct sets of rules and how the App Store on iOS does not even have a dedicated “Developer Tools” section.

I have been on a call with the App Review Team at Apple twice. I was told that Windmill on the iPhone is “not acceptable” to be on the App Store. Additionally, taking a “holistic” look at Windmill I was told that “conceptually” it is in violation of the Apple Developer Program License Agreement.

I also emailed an Apple Developer Tools Evangelist asking for help to find a way forward with Windmill and for some course correction to be done. That was more than 2 weeks ago. I haven’t received a reply.

It’s been really hard to come to terms with this. I fought really hard for Windmill.

I both understand where Apple is coming from and I don’t.

Update

I was reminded of one very specific reason that I was given too.

Guideline 5.2.5 - Legal - Intellectual Property

YOUR APP IS TOO SIMILAR TO TESTFLIGHT, WHICH CREATES A MISLEADING ASSOCIATION WITH APPLE PRODUCTS. WE ENCOURAGE YOU TO REVIEW YOUR APP CONCEPT AND EVALUATE WHETHER YOU CAN INCORPORATE DIFFERENT CONTENT AND FEATURES TO BRING IT INTO COMPLIANCE WITH THE APP STORE REVIEW GUIDELINES.

I understand

Apple seems to take issue with the fact that Windmill is meant to be used for development, in-house, but Windmill on the iPhone was meant to be made available on the App Store.

More importantly. Apple took the stance that the Command Line Tools Package is only meant to be used by developers in-house and not by 3rd parties to provide support for continuous integration systems - continuous delivery in the case of Windmill.

Before you gasp, here is the nuance as I understand it1.

Apple takes issue that Windmill makes it effortless for developers to create a distribution mechanism for their apps outside the control of Apple and the safety of the App Store and TestFlight.

Hosting of pre-release software through means outside of the app store and facilitating developers to use their certificate, provisioning profile to do so is not permitted.

Even though ad-hoc distribution is limited, Windmill imposes strong security requirements and is designed for in-house use only.

After all, Windmill is available to every developer. Not just organisations with the know-how. Its mission is to make continuous delivery of iPhone apps ubiquitous and accessible to developers and businesses in an effort to improve consistency, a high level of understanding and agreement across the industry.

It doesn’t seem to matter what Windmill is designed to be used for but what it could be used for. It is precisely why Apple had to say that the Command Line Tools Package is only meant to be used by developers alone.

Xcode command line tools are only provided by Apple to developers to automate building, testing and distributing their app.

Had an organisation built Windmill for internal use, distribution would be under their control, used exclusively for their apps, and operating in their internal network.

Windmill on the iPhone would probably be built and distributed using an Enterprise Certificate, internally to the employees of the organisation or using ad-hoc distribution. In this case, it wouldn’t matter since it’s internal to the organisation.

Even Xcode Server, Apple’s home grown offering to in-house continuous integration and distribution comes bundled with a web server, “where you and members of your development team can use a web browser to view the status of bot integrations and download assets and products.” See Monitor and Manage Bots.

I don’t understand

Windmill does not operate outside of the boundaries as defined by Apple.

  • It expects that an organisation is registered with the Apple Developer Program.
  • It expects an installation of the tools in any development Mac where Windmill is used. Windmill itself does not come bundled with the Command Line Tools Package.
  • It manages signing based on the Apple Developer Team of the developer, organisation.
  • It uses whatever Distribution Certificate and Provisioning Profile is managed by Xcode.
  • It is limited by ad-hoc distribution device limits.

Windmill could not be used to distribute apps to the public, circumventing TestFlight and the App Store anymore than a developer can do today without Windmill or using existing services that offer ad-hoc distribution on the web.

Windmill plays by every Apple rule. It does not ask a for Apple credentials. It does not manage signing outside of Xcode. It does not ask for the private key and distribution certificate. It does everything that a developer could do today, with better security and privacy than an equivalent service on the cloud. There is a whole industry providing automation and distribution as a service for Apple platforms on the web.

Windmill may be the only third-party native macOS continuous delivery app available on Apple platforms but it probably comes last in offering such a service to developers.

Reconciliation

I wish Windmill on the Mac was also on the App Store. Reviewed, approved, endorsed and recommended by Apple. The decision to build Windmill on macOS and iOS has become a liability.

With Windmill, I effectively asked Apple to weigh in on the matter and Apple has decided that the risk far outweighs the benefits that Windmill has compared to existing offerings outside the Apple ecosystem.

It is possible that Windmill can only live within the confines of the Apple Business Manager with Windmill on the iPhone as a Custom App. Which does not help fulfil its mission.

Collectively as an iOS community, we have spent an insane amount of time and put an immense amount of effort in automating the delivery of our apps by writing flaky scripts in bash using Jenkins, reverse engineering the Xcode project file format, writing blog posts with our findings. Yet we are none the wiser and have barely made a dent. - iOS Automation - The Current State of Affairs

I completely misread the situation with Apple. It does not make it any easier that Apple has a tendency to be overly protective of the ecosystem. I understand why, I just wish Apple had embraced Windmill.

I had made plans to take Windmill even further, beyond distribution, onto other Apple platforms. Now, I am not sure what I can do while Apple stays silent.

One more thing

Windmill is the culmination of the experience I have accumulated as a developer over the past 15 or so years. On top of that, it took months of research and development on the Mac, on the server, and on the iPhone, paying great attention to the user experience as well as security and privacy that would not and cannot be made possible without deeply integrating with Apple’s tools, platforms and services.

I can’t help but feel disappointed and highly discouraged after all the work I’ve put in improving the tooling available to developers.

I would like to express my deepest gratitude to everyone at the Seattle Xcoders group. They have been very supportive all this time when I struggled mentally and needed them the most.

Thank you Brent, Jared, Tom, Tim, Marc, Jeff for providing invaluable feedback, improving this post significantly in the process.

Special thanks to Daniel Jalkut.

A call for discussion

Beyond the personal and financial setback this is causing me, I am hearing from the developer community there is a broader concern around the restrictions and the limitation on the tools Apple provides and allows.

I want to kick-off a discussion about this to address these shortcomings. How should Apple enable developers to improve on the tooling for the iOS development ecosystem? I’d love to hear your thoughts.

Related

  1. Apple didn’t really explain their reasoning. ↩︎

What is this website?

This is a personal website, at the outskirts of the web, away from social media and publishing platforms. This website surfaces social, racial, economic traits and explores human relationships. It highlights the conditions that contribute to one's personal success or downfall. It shares stories that act as a reminder that life is messy, complex, nuanced, diverse. It aims to bring the world closer together. It reaches out to those that feel lost, lonely, inadequate and outcasts. I am with you.

for historic purposes, 20th to 21st century