Realize various operations on images (UTexture2D) (read, save, copy, clipboard...)

The following code examples are based on version UE5.3.

Source code

More source code details can be obtained from the UE Marketplace plugin: AIChatPlus

Read: UE realizes reading local system images as UTexture2D

General method

This method is feasible in both Editor and GamePlay modes, supporting image file formats including PNG, JPEG, BMP, ICO, EXR, ICNS, HDR, TIFF, DDS, TGA, covering most common image types.

The code is also very clean:

#include <Engine/Texture2D.h>
#include <ImageUtils.h>

UTexture2D* LoadImage(const FString& InLoadPath)
{
    FImage ImageInfo;
    FImageUtils::LoadImage(*InLoadPath, ImageInfo);
    return FImageUtils::CreateTexture2DFromImage(ImageInfo);
}

The returned is UTexture2D.

Editor-specific methods

This method can additionally support more image types: UDIM texture maps, IES files, PCX, PSD.

The implementation of the code will be more complex:

#include <Engine/Texture2D.h>
#include <Misc/FileHelper.h>
#include <Misc/Paths.h>
#include <UObject/UObjectGlobals.h>

#if WITH_EDITOR
UTexture2D* LoadImage(const FString& InLoadPath)
{

    TArray64<uint8> Buffer;
    if (!FFileHelper::LoadFileToArray(Buffer, *InLoadPath))
    {
        return nullptr;
    }

    const FString TextureName;
    const FString Extension = FPaths::GetExtension(InLoadPath).ToLower();
    const uint8* BufferPtr = Buffer.GetData();

    auto TextureFact = NewObject<UTextureFactory>();
    UTexture2D* Ret = Cast<UTexture2D>(TextureFact->FactoryCreateBinary(
        UTexture2D::StaticClass(), GetTransientPackage(), *TextureName, RF_Transient,
        NULL, *Extension, BufferPtr, BufferPtr + Buffer.Num(), GWarn));

    return Ret;
}
#endif

The implementation uses the FactoryCreateBinary function of UTextureFactory, which is able to read the additional file types mentioned earlier.

Copy: UE achieves copying UTexture2D

Sometimes you need to duplicate a UTexture2D and then modify the duplicated image. Copying the image requires using the engine's built-in function FImageCore::CopyImage. Just set the parameters for the two images and call this interface.

UTexture2D* CopyTexture2D(UTexture2D* InTexture, UObject* Outer, FName Name, EObjectFlags Flags)
{
    // src texture info, src ImageView
    FTextureMipDataLockGuard InTextureGuard(InTexture);
    uint8* SrcMipData = InTextureGuard.Lock(LOCK_READ_ONLY);        // Texture->GetPlatformData()->Mips[0].BulkData.Lock(InLockFlag)
    const int32 InSizeX = InTexture->GetSizeX();
    const int32 InSizeY = InTexture->GetSizeY();
    const EPixelFormat InFormat = InTexture->GetPixelFormat();
    const FImageView SrcMipImage(
        SrcMipData, InSizeX, InSizeY, 1, GetRawImageFormat(InFormat), InTexture->GetGammaSpace());

    // create dst texture
    UTexture2D* NewTexture = NewObject<UTexture2D>(Outer, Name, Flags);
    NewTexture->SetPlatformData(new FTexturePlatformData());
    NewTexture->GetPlatformData()->SizeX = InSizeX;
    NewTexture->GetPlatformData()->SizeY = InSizeY;
    NewTexture->GetPlatformData()->SetNumSlices(1);
    NewTexture->GetPlatformData()->PixelFormat = InFormat;

    // Allocate first mipmap.
    int32 NumBlocksX = InSizeX / GPixelFormats[InFormat].BlockSizeX;
    int32 NumBlocksY = InSizeY / GPixelFormats[InFormat].BlockSizeY;
    FTexture2DMipMap* Mip = new FTexture2DMipMap();
    Mip->SizeX = InSizeX;
    Mip->SizeY = InSizeY;
    Mip->SizeX = 1;
    NewTexture->GetPlatformData()->Mips.Add(Mip);
    Mip->BulkData.Lock(LOCK_READ_WRITE);
    Mip->BulkData.Realloc((int64)NumBlocksX * NumBlocksY * GPixelFormats[InFormat].BlockBytes);
    Mip->BulkData.Unlock();

    // dst texture ImageView
    uint8* DstMipData = static_cast<uint8*>(NewTexture->GetPlatformData()->Mips[0].BulkData.Lock(LOCK_READ_WRITE));
    const FImageView DstMipImage(
        DstMipData, InSizeX, InSizeY, 1, GetRawImageFormat(InFormat), InTexture->GetGammaSpace());

    // run CopyImage
    FImageCore::CopyImage(SrcMipImage,DstMipImage);

#if WITH_EDITORONLY_DATA
    NewTexture->Source.Init(
        InSizeX, InSizeY, 1, 1,
        FImageCoreUtils::ConvertToTextureSourceFormat(GetRawImageFormat(InFormat)), DstMipData);
#endif

    // cleanup
    NewTexture->GetPlatformData()->Mips[0].BulkData.Unlock();
    NewTexture->UpdateResource();

    return NewTexture;
}

Save: UE achieves saving UTexture2D to file

The key is to use the engine function FImageUtils::SaveImageAutoFormat, which is relatively simple to implement, but it is important to pay attention to the situation of retrying in case of failure.

void SaveImage(UTexture2D* InImage, const FString& InSavePath)
{
    if (!InImage) return;
    FImage ImageInfo;
    if (FImageUtils::GetTexture2DSourceImage(InImage, ImageInfo))
    {
        FImageUtils::SaveImageAutoFormat(*InSavePath, ImageInfo);
    }
    else
    {
        // if prev save failed
        // use ConvertTextureToStandard to change InImage to Standard format, and try again
        // then revert InImage's origin format
        // this is what FTextureMipDataLockGuard does
        FTextureMipDataLockGuard InImageGuard(InImage);

        uint8* MipData = InImageGuard.Lock(LOCK_READ_ONLY);
        check( MipData != nullptr );

        const FImageView MipImage(
            MipData, InImage->GetSizeX(), InImage->GetSizeY(), 1,
            GetRawImageFormat(InImage->GetPixelFormat()), InImage->GetGammaSpace());

        FImageUtils::SaveImageAutoFormat(*InSavePath, MipImage);
    }
}

Save: Implement saving UTexture2D to Asset

Save the UTexture2D in memory to an Asset, and be able to view it in the Content Browser.

The core function needs to use the CopyTexture2D implemented above. We need to first duplicate a new image, and then call UPackage::SavePackage to save the Package where the image is located as an Asset.

void SaveTextureToAsset(UTexture2D* InTexture)
{
    if (!InTexture) return;

    // open save asset dialog, choose where/which to save
    FSaveAssetDialogConfig SaveAssetDialogConfig;

    SaveAssetDialogConfig.DefaultPath =  FEditorDirectories::Get().GetLastDirectory(ELastDirectory::NEW_ASSET);
    SaveAssetDialogConfig.AssetClassNames.Add(UTexture2D::StaticClass()->GetClassPathName());
    SaveAssetDialogConfig.ExistingAssetPolicy = ESaveAssetDialogExistingAssetPolicy::AllowButWarn;
    SaveAssetDialogConfig.DialogTitleOverride = FAIChatPlusEditor_Constants::FCText::SaveAsAsset;

    const FContentBrowserModule& ContentBrowserModule = FModuleManager::LoadModuleChecked<FContentBrowserModule>("ContentBrowser");
    const FString SaveObjectPath = ContentBrowserModule.Get().CreateModalSaveAssetDialog(SaveAssetDialogConfig);

    if (SaveObjectPath.IsEmpty()) return;

    // init save info
    const FString PackageName = FPackageName::ObjectPathToPackageName(SaveObjectPath);
    const FString PackageFileName = FPackageName::LongPackageNameToFilename(PackageName, FPackageName::GetAssetPackageExtension());
    const FString PackagePath = FPaths::GetPath(PackageFileName);
    const FString TextureName = FPaths::GetBaseFilename(PackageName);

    // create new UPackage to put the new texture in
    UPackage* const NewPackage = CreatePackage(*PackageName);
    NewPackage->FullyLoad();

    // copy texture
    UTexture2D* NewTexture = UAIChatPlus_Util::CopyTexture2D(
        InTexture, NewPackage, FName(TextureName), RF_Public | RF_Standalone | RF_Transactional);

    // Generate the thumbnail
    // if not doing so, the texture will not have thumbnail in content browser
    FObjectThumbnail NewThumbnail;
    ThumbnailTools::RenderThumbnail(
        NewTexture, NewTexture->GetSizeX(), NewTexture->GetSizeY(),
        ThumbnailTools::EThumbnailTextureFlushMode::NeverFlush, NULL,
        &NewThumbnail);
    ThumbnailTools::CacheThumbnail(NewTexture->GetFullName(), &NewThumbnail, NewPackage);

    // setting up new package and new texture
    NewPackage->MarkPackageDirty();
    FAssetRegistryModule::AssetCreated(NewTexture);
    FEditorDirectories::Get().SetLastDirectory(ELastDirectory::NEW_ASSET, FPaths::GetPath(PackageName));

    // save args
    FSavePackageArgs SaveArgs;
    SaveArgs.TopLevelFlags = RF_Public | RF_Standalone;
    SaveArgs.bForceByteSwapping = true;
    SaveArgs.bWarnOfLongFilename = true;

    // save it
    if (!UPackage::SavePackage(NewPackage, NewTexture, *PackageFileName, SaveArgs))
    {
        UE_LOG(AIChatPlusEditor, Error, TEXT("Failed to save Asset: [%s]\n"), *PackageFileName);
    }
}

Clipboard: UE implement copying images (UTexture2D) to Windows Clipboard

Windows related functions

We will use the following functions related to the Windows clipboard operations:

• OpenClipboardOpen the clipboard to obtain the clipboard's handler.
• EmptyClipboardEmpty the clipboard and assign ownership of the clipboard to the current window.
• SetClipboardDataSet the data of the clipboard. Image data is sent to the clipboard through this interface.
• CloseClipboardAfter setting up the data, close the clipboard.

Clipboard image format

(https://learn.microsoft.com/zh-cn/windows/win32/dataxchg/standard-clipboard-formats)Inside, there are introductions to available clipboard formats, among which CF_DIBV5 can be used to set images.

The specific definition of the format required by CF_DIBV5 BITMAPV5HEADER structureHere we have selected the following configuration.

BITMAPV5HEADER Header;
Header.bV5CSType        = LCS_sRGB;
Header.bV5Compression   = BI_BITFIELDS;

UTexture2D Setting

We selected the color space of the clipboard image as LCS_sRGB, which is the sRGB color space above, so the UTexture2D also needs to be set to the corresponding format first:

bool ConvertTextureToStandard(UTexture2D* InTexture)
{
    if (InTexture->CompressionSettings != TC_VectorDisplacementmap)
    {
        InTexture->CompressionSettings = TC_VectorDisplacementmap;
        IsChanged = true;
    }
    if (InTexture->SRGB != true)
    {
        InTexture->SRGB = true;
        IsChanged = true;
    }
    if (IsChanged)
    {
        InTexture->UpdateResource();
    }
}

ConvertTextureToStandard is responsible for converting UTexture2D to standard format: TC_VectorDisplacementmap (RGBA8) and SRGB color space. Once aligning the image format of UTexture2D and Windows clipboard, we can copy the image data to the clipboard.

Specific code

void CopyTexture2DToClipboard(UTexture2D* InTexture)
{
    if (!InTexture) return;

    FTextureMipDataLockGuard InTextureGuard(InTexture);
    // get InTexture info
    uint8* SrcMipData = InTextureGuard.Lock(LOCK_READ_ONLY);
    const int32 InSizeX = InTexture->GetSizeX();
    const int32 InSizeY = InTexture->GetSizeY();
    const EPixelFormat InFormat = InTexture->GetPixelFormat();
    const FImageView SrcMipImage(
        SrcMipData, InSizeX, InSizeY, 1, GetRawImageFormat(InTexture), InTexture->GetGammaSpace());

    // set clipboard Texture info
    const EPixelFormat OutFormat = PF_B8G8R8A8;
    const int32 NumBlocksX = InSizeX / GPixelFormats[OutFormat].BlockSizeX;
    const int32 NumBlocksY = InSizeY / GPixelFormats[OutFormat].BlockSizeY;
    const int64 BufSize = static_cast<int64>(NumBlocksX) * NumBlocksY * GPixelFormats[InFormat].BlockBytes;

    // set header info
    BITMAPV5HEADER Header;
    Header.bV5Size          = sizeof(BITMAPV5HEADER);
    Header.bV5Width         = InSizeX;
    Header.bV5Height        = -InSizeY;
    Header.bV5Planes        = 1;
    Header.bV5BitCount      = 32;
    Header.bV5Compression   = BI_BITFIELDS;
    Header.bV5SizeImage     = BufSize;
    Header.bV5XPelsPerMeter = 0;
    Header.bV5YPelsPerMeter = 0;
    Header.bV5ClrUsed       = 0;
    Header.bV5ClrImportant  = 0;
    Header.bV5RedMask       = 0x00FF0000;
    Header.bV5GreenMask     = 0x0000FF00;
    Header.bV5BlueMask      = 0x000000FF;
    Header.bV5AlphaMask     = 0xFF000000;
    Header.bV5CSType        = LCS_sRGB;
    // Header.bV5Endpoints;    // ignored
    Header.bV5GammaRed      = 0;
    Header.bV5GammaGreen    = 0;
    Header.bV5GammaBlue     = 0;
    Header.bV5Intent        = 0;
    Header.bV5ProfileData   = 0;
    Header.bV5ProfileSize   = 0;
    Header.bV5Reserved      = 0;

    HGLOBAL WinBuf = GlobalAlloc(GMEM_MOVEABLE, sizeof(BITMAPV5HEADER) + BufSize);
    if (WinBuf == NULL)
        return;

    HWND WinHandler = GetActiveWindow();
    if (!OpenClipboard(WinHandler)) {
        GlobalFree(WinBuf);
        return;
    }
    verify(EmptyClipboard());

    // copy InTexture into BGRA8 sRGB Standard Texture
    FTexture2DMipMap* DstMip = new FTexture2DMipMap();
    DstMip->SizeX = InSizeX;
    DstMip->SizeY = InSizeY;
    DstMip->SizeZ = 1;
    DstMip->BulkData.Lock(LOCK_READ_WRITE);
    uint8* DstMipData = static_cast<uint8*>(DstMip->BulkData.Realloc(BufSize));
    const FImageView DstMipImage(
        DstMipData, InSizeX, InSizeY, 1, ERawImageFormat::BGRA8, EGammaSpace::sRGB);

    FImageCore::CopyImage(SrcMipImage,DstMipImage);
    DstMip->BulkData.Unlock();

    // copy Standard Texture data into Clipboard
    void * WinLockedBuf = GlobalLock(WinBuf);
    if (WinLockedBuf) {
        memcpy(WinLockedBuf, &Header, sizeof(BITMAPV5HEADER));
        memcpy((char*)WinLockedBuf + sizeof(BITMAPV5HEADER), DstMipData, BufSize);
    }
    GlobalUnlock(WinLockedBuf);

    if (!SetClipboardData(CF_DIBV5, WinBuf))
    {
        UE_LOG(AIChatPlus_Internal, Fatal, TEXT("SetClipboardData failed with error code %i"), (uint32)GetLastError() );
    }

    // finish, close clipboard
    verify(CloseClipboard());

    delete DstMip;
}

Conversion between UTexture2D and Base64

This is relatively easy to implement, let's dive into the code.

#include <Misc/Base64.h>
#include <ImageUtils.h>

UTexture2D* B64ToImage(const FString& B64)
{
    TArray<uint8> Data;
    FBase64::Decode(B64, Data);
    return FImageUtils::ImportBufferAsTexture2D(Data);
}

FString ImageToB64(UTexture2D* InTexture, const int32 InQuality)
{
    FTextureMipDataLockGuard InTextureGuard(InTexture);

    uint8* MipData = InTextureGuard.Lock(LOCK_READ_ONLY);
    check(MipData != nullptr);

    const FImageView InImage(
        MipData, InTexture->GetSizeX(), InTexture->GetSizeY(), 1,
        GetRawImageFormat(InTexture->GetPixelFormat()), InTexture->GetGammaSpace());

    TArray64<uint8> Buffer;
    FString Ret;
    if (FImageUtils::CompressImage(Buffer, TEXT("png"), InImage, InQuality))
    {
        Ret = FBase64::Encode(Buffer.GetData(), Buffer.Num());
    }
    return Ret;
}

Original: https://wiki.disenone.site/en

This post is protected by CC BY-NC-SA 4.0 agreement, should be reproduced with attribution.

Visitors. Total Visits. Page Visits.

This post is translated using ChatGPT, please feedback if any omissions.