This can be done by simply using “//url” instead of “http://url”. As it stands, any page loaded via https with the maps does not show as secure because some elements are not loaded securely.
Although I just realized that trying to load the current opencyclemaps images via https does not come in as secure. Maybe there are secure urls available?